HPCPerfStats icon indicating copy to clipboard operation
HPCPerfStats copied to clipboard

Data aggregation for shorter sample times

Open stephenlienharrell opened this issue 2 years ago • 1 comments

We should explore data aggregation to get our sample times much lower.

It is my thought to keep the data-points at 5 minutes, but store average/min/max/std over 10 samples (30 second sampling).

This would only increase the data by 4x, instead of 10x.

then we could graph with the average and min/max bars to give a more complete picture on what happened on the node.

We probably want to do this on-node.

stephenlienharrell avatar Jun 21 '23 18:06 stephenlienharrell

really a data reduction problem only, sampling size

stephenlienharrell avatar Aug 23 '23 16:08 stephenlienharrell