server icon indicating copy to clipboard operation
server copied to clipboard
verified publisher icon triton-inference-server

Low QPS with momentary traffic surges cause significant increases in inference TP99 latency.

Open a1342772 opened this issue 1 year ago • 1 comments

Description Low QPS with momentary traffic surges cause significant increases in inference TP99 latency.

Triton Information 23.06

31

企业微信截图_f293c4aa-fc1e-4939-b7a9-02f102fb2be0

a1342772 avatar Jun 03 '24 08:06 a1342772

@a1342772 We would need detailed steps to reproduce this issue to investigate further. Kindly provide these details

statiraju avatar Jun 11 '24 17:06 statiraju

Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.

krishung5 avatar Aug 26 '24 22:08 krishung5