[Question] About perf_analyzer request rate

[SeungsuBaek]

Hi!

I have a question about perf_analyzer and tritonserver's trace.json.

---

SERVER

$ tritonserver --model-repository=/triton_dev/models --trace-file /triton_dev/experiments/trace/trace.json --trace-level TIMESTAMPS --trace-rate 1

---

CLIENT

$ perf_analyzer -b 1 -m resnet --request-rate-range 50:50 --sync --request-distribution constant

I wanted to see if the request comes to the server according to the request rate set by the perf_analyzer. So after using these commands, I checked trace.json and drew a graph of how many requests per second arrived.

Below graph is the result of that. ( { "name": "http recv start", "ns": 123456789} ) // only use this information in trace.json.

My question is that why is the request rate measured higher than the perf_analyzer's value for about 3 seconds?

[GuanLuo]

@tanmayv25 @matthewkotila @Tabrizian ^^^

[tanmayv25]

There appears to be a bug. PA generates a schedule for requests to follow. Most likely the initial few requests have missed their scheduled time and get sent together. @matthewkotila Can you create a bug to fix this issue?

[tgerdesnv]

Hi @SeungsuBaek, there have been some fixes around request rate scheduling over the last few months. I tried to reproduce this issue and everything appears to be working correctly now. Let me know if you are still seeing this issue.

[SeungsuBaek]

@tgerdesnv Thanks you so much!!!