Kris Hung comments

Results 129 comments of


                                            Kris Hung

trafficstars

triton-inference-server cannot be started

Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.

tensortllm backend fails when kv cache is disabled

Looks duplicated to this issue in TRT-LLM repo: https://github.com/NVIDIA/TensorRT-LLM/issues/2443/

gpt_model_path with Triton's S3 based model repository support

I think the `TRITON_MODEL_DIRECTORY` is only supported in Triton Python backend at the moment. It is up to the backend to be able to dynamically set the path when using...

could you give some examples about ragged input config for tensorrt backend

Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.

Why is my model in ensemble receiving out-of-order input

Hi @Joenhle, thanks for fling the issue. I was wondering which version of Triton do you use? Can you please provide a minimum reproducer for us to investigate? The order...

Why is my model in ensemble receiving out-of-order input

Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.

Tritonserver hangs on launch with python backend

Hi @JamesBowerXanda, Triton doesn't officially support Mac, but I assume it would work if you are only running CPU-only model. I couldn't reproduce the hang using a linux machine. Since...

Tritonserver hangs on launch with python backend

Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.

Triton Tensorrt-LLM 24.04 and 24.05 are very large

Hi @yaysummeriscoming, the TRT-LLM container has different dependency stack. Some packages like pytorch are required during runtime so the image size is larger than the other Triton images. We are...