tensorrtllm_backend
tensorrtllm_backend copied to clipboard
fill_template.py and gpu_device_ids
System Info
3090
Who can help?
@juney-nvidia @byshiue @schetlur-nv
Information
- [ ] The official example scripts
- [X] My own modified scripts
Tasks
- [X] An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below)
Reproduction
I want to use both gpu_device_ids 0 and 1:
python3 fill_template.py --in_place config.pbtxt gpu_device_ids:0,1
Expected behavior
work
actual behavior
Traceback (most recent call last):
File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
main(**vars(args))
File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
main(**vars(args))
File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
main(**vars(args))
File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
main(**vars(args))
File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
main(**vars(args))
File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
additional notes
the problem is "," in "0,1"