Prashanth Nandavanam comments

Repositories
Issues
Comments

Results 3 comments of


                                            Prashanth Nandavanam

Prebuilt Triton Server 24.05-trtllm-python-py3 does not have correct TensorRT version

I've run into this myself. I'm attempting to deploy LLama3 and Gemma, and keep running into these issues when generating the engines. Will there be an update/fix any time soon?...

Prebuilt Triton Server 24.05-trtllm-python-py3 does not have correct TensorRT version

@geraldstanje - I JUST got gemma to deploy (yet to test), after much trial and error. Getting to the correct version number combination for Triton, TensorRT, TensorRT-LLM, and tensorrt_llm_backend involved...

Prebuilt Triton Server 24.05-trtllm-python-py3 does not have correct TensorRT version

Thanks, @CarterYancey - good to know I wasn't the only one suffering. I did make sure the versions were the same. As you pointed out, the documentation is not accurate,...