juney-nvidia
juney-nvidia
@Pevernow Can you elaborate more about your request? Thanks June
Sorry for replying late due to being trapped by other things. > Users want something like this https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md, so they can switch their apps from OpenAI models to TRT-LLM models...
Thanks for your suggestion. Let me add it to the list of models that were requested and we will keep you posted. Juney
@Muhtasham can you share the concrete command sequence to reproduce the issue? Including how you build the engine. Thanks June
@RalphMao do you have any comments on this ask? :)
@Zars19 thanks for the contribution to TensorRT-LLM! @nv-guomingz can you help take care of this? :) Thanks June
@pathorn Hi Pathorn Thanks for your interest to submit the MR into TRT-LLM. The current process of merging community MR into TRT-LLM is: - After the contributor finishing the implementation...
@wjj19950828 Hi, can you follow [this](https://github.com/triton-inference-server/tensorrtllm_backend/issues/270) template to provide the concrete steps to reproduce your issue? Then our engineers can help with the investigation. June
@matichon-vultureprime Thanks for reporting this. Currently the ARM support of TRT-LLM is still at experimental phase, so it may contain issues. When the ARM support is stable enough, we will...
Thanks for preparing the MR, @syuoni ! June