tensorrtllm_backend
tensorrtllm_backend copied to clipboard
llm performance metric
tensorrtllm have the metric of llm performance , e.g TTFT, latency of token , input token, output token。 if have, please tell me; if not , can add the metric like vllm metric。the metric is very important。