Benchmark client metrics

[JakubDaleki]

Hi,

I run benchmark client in python to measure performance of model server, but I am a bit confused by the meaning of various metrics e.g. 'worker: mean_latency', 'worker: mean_latency2' or 'worker: window_cv_latency'. I couldn't find but is there any documentation for each of these?

Thanks in advance,

Jakub

[mzegla]

Hi,

Did you have a chance to take a look at this doc: https://github.com/openvinotoolkit/model_server/blob/releases/2022/1/docs/python-benchmarking-client-16feb.pdf ?

@krzyczar is the author of the client and above document, so hopefully he can clarify any doubts.

[krzyczar]

Hi Jakub, @JakubDaleki

window_cv_latency - coefficient of variation under window for latency calculated as:

CV= μ / σ where: σ=standard deviation μ=mean ​ mean_latency2 - is average of squared latency - we use it to calculate standard deviation / variance - E[lat^2] - (E[lat])^2

no prefixes "window" and "warmup" means that it is for the whole workload

We documented only selected, most important things, therefore you will not find CV/mean2 in documentation yet.

best regards Krzysztof

[JakubDaleki]

Thank you.