TensorRT-LLM feat: Add support of chat completion in PD

Add support of chat completion in PD and fix the include_usage option.

Mar 23 '25 14:03 Shunkangz

/bot run

Mar 23 '25 14:03 Shunkangz

@chuangz0 @xiaoweiw-nv @pcastonguay pls help review this MR.

Thanks June

Mar 23 '25 22:03 juney-nvidia

/bot run

Mar 24 '25 08:03 Shixiaowei02

PR_Github #269 [ run ] triggered by Bot

Mar 24 '25 08:03 niukuo

PR_Github #269 [ run ] completed with state FAILURE /LLM/main/L0_MergeRequest_PR pipeline #259 completed with status: 'FAILURE'

Mar 24 '25 12:03 niukuo

There are no tests for the chat endpoint. Could we add one please? Thanks.

Sure. I think that the modification of chat endpoint should be orthogonal to the parallel strategy of context and generation workers. Do we need to double all existing tests for chat or just add single test should be enough? @pcastonguay

Mar 26 '25 06:03 Shunkangz

There are no tests for the chat endpoint. Could we add one please? Thanks.

Sure. I think that the modification of chat endpoint should be orthogonal to the parallel strategy of context and generation workers. Do we need to double all existing tests for chat or just add single test should be enough? @pcastonguay

Single test should be enough.

Mar 26 '25 13:03 pcastonguay

/bot run --add-multi-gpu-test

Apr 08 '25 09:04 Shunkangz

PR_Github #1438 [ run ] triggered by Bot

Apr 08 '25 09:04 tensorrt-cicd

PR_Github #1438 [ run ] completed with state SUCCESS /LLM/main/L0_MergeRequest_PR pipeline #1079 completed with status: 'FAILURE'

Apr 08 '25 11:04 tensorrt-cicd

/bot run --add-multi-gpu-test

Apr 09 '25 14:04 Shunkangz

PR_Github #1629 [ run ] triggered by Bot

Apr 09 '25 15:04 tensorrt-cicd

/bot run --add-multi-gpu-test

Apr 10 '25 01:04 Shunkangz

PR_Github #1668 [ run ] triggered by Bot