tvm
tvm copied to clipboard
[TVMC] Allow selecting a subset of tasks to be used in `tvmc tune`
This adds a --tasks
flag to the tvmc tune
command to filter the lists of tasks to be tuned. See examples below.
Motivation
- As auto-tuning can be quite time consuming, it is often desirable to cut down the number of tuned tasks in a session.
- If the tuning session was canceled halfway through, it would be a bad idea to start from scratch. Instead continue with the last untuned task
- Some tasks have more impact on the model performance than others, thus we should be able to train some tasks longer than others
Examples
- Use
--task list
to show which tasks are available for tuning
$ tvmc tune toycar.tflite -o out.txt --task list
Available Tasks for tuning:
0. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640), 'int...
1. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640), 'int16...
2. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (128, 128), 'int...
3. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (128, 128), 'int16...
4. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (8, 128), 'int16...
5. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (8, 128), 'int16')...
6. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 8), 'int16'), ('TENSOR', (128, 8), 'int16')...
7. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 8), 'int16'), ('TENSOR', (128, 8), 'int16'), ...
8. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (640, 128), 'int...
9. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 128), 'int16'), ('TENSOR', (640, 128), 'int16...
- Filter the list of tasks to be tuned:
# Only tune a single task (index 5)
tvmc tune toycar.tflite -o out.txt --tasks 5
# Tunes tasks starting with index 6
tvmc tune toycar.tflite -o out.txt --tasks "6-"
# Tune tasks 1,4,5,6,8,9
tvmc tune toycar.tflite -o out.txt --tasks "1,4-6,8-"
Tests
I added a basic unit test for the filter_tasks
utility in tests/python/driver/tvmc/test_autotuner.py
.
Open Questions
- ~~While the (truncated) string representations of AutoTVM tasks are quite helpful to pick the correct tasks, using AutoScheduler the tasks can not really be distinguished from each other (only by index). Is there a way to get similar information from AutoScheduler tasks?~~
cc @Mousius @gromero
CC @leandron @giuseros @areusch @tqchen
@PhilippvK i think this one needs a rebase
Short update:
- With my latest changes I can now print useful task names for AutoScheduler as well (using
task.desc
instead ofstr(task)
) - I added a mention of the
--task
argument in the TVMC CLI tutorial
@areusch
As I have to retrigger the CI anyway so I want to propose another small addition:
For AutoTVM, in addition to the task description, I could also print len(task.config_space)
to give an idea on how large the tuning search space is for a specific task, which can help a lot to estimate the required number of tuning trials.
Example:
Available Tasks for tuning:
0. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640)... (len=640)
1. Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640), ... (len=35)
Do you think this would be a good addition to tvmc tune --task list
?
@areusch Can I have an update on this?
@tvm-bot rerun
sorry @PhilippvK I missed this one. feel free to ping me on discord if that happens again. I retriggered the CI, I'm good for @leandron to merge this if it passes.
Example:
Available Tasks for tuning: 0. Task(func_name=dense_nopack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640)... (len=640)
- Task(func_name=dense_pack.x86, args=(('TENSOR', (1, 640), 'int16'), ('TENSOR', (128, 640), ... (len=35) Do you think this would be a good addition to tvmc tune --task list?
Sure that seems like a good addition!
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
- Built docs for commit ff33ca0c1d2bd031a4c05a13be0422533f4de3a7 can be found here.
Generated by tvm-bot
Rebased and CI should be fixed now @leandron @areusch.
Sorry @PhilippvK. It took me a while to catch up with this. Can you rebase and then we move to get this merged?
@leandron Rebased on upstream/main
@tvm-bot rerun
Rebased ontop of main
and CI errors seem to be fixed. Can we merge this @areusch @leandron ?
Any further comments @areusch @leandron?
Rebased to fix conflict + I added 2 small AutoTVM related improvements:
- If #trials is less then, #tasks, tune each task at least once, instead of failing all of them.
- If the config space of a single task is smaller than the determined number of trials per task, use the smaller values as bound in the progress bar.
@tvm-bot rerun
@lhutton1 Hey, I implemented the proposed changes and the CI passed. Feel free to merge after quickly having a look at the refactored unit tests.
Thanks @PhilippvK @areusch @leandron!