tensorrtllm_backend
tensorrtllm_backend copied to clipboard
Other backends are missing
System Info
rtx4090
Who can help?
@
Information
- [ ] The official example scripts
- [ ] My own modified scripts
Tasks
- [ ] An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below)
Reproduction
I use this docker image: nvcr.io/nvidia/tritonserver:24.04-trtllm-python-py3, however, when I run and enter container, there are two backends:
why?
Expected behavior
Normal backends:
actual behavior
additional notes
None