tensorrtllm_backend icon indicating copy to clipboard operation
tensorrtllm_backend copied to clipboard

Is it possible to edit backend within config.pbtxt from python backed to tensorrtllm backen, Whisper model ?

Open rungrodkspeed opened this issue 1 year ago • 0 comments

I do this practice : https://github.com/k2-fsa/sherpa/tree/master/triton/whisper and it's good working.

I want to know if change backed parameter in config.pbtxt from python to tensorrtllm, performance maybe better ?

Do I need to change any steps in the convert whisper to tensorrtllm which can perform on tensorrtllm backend ?

This is result after change parameter backend from python to tensorrtllm: image

P.S. I use NGC: nvcr.io/nvidia/tritonserver:24.07-trtllm-python-py3

rungrodkspeed avatar Sep 26 '24 07:09 rungrodkspeed