tensorrtllm_backend
tensorrtllm_backend copied to clipboard
Is it possible to edit backend within config.pbtxt from python backed to tensorrtllm backen, Whisper model ?
I do this practice : https://github.com/k2-fsa/sherpa/tree/master/triton/whisper and it's good working.
I want to know if change backed parameter in config.pbtxt from python to tensorrtllm, performance maybe better ?
Do I need to change any steps in the convert whisper to tensorrtllm which can perform on tensorrtllm backend ?
This is result after change parameter backend from python to tensorrtllm:
P.S. I use NGC: nvcr.io/nvidia/tritonserver:24.07-trtllm-python-py3