Summary stats (Averages/P95/P99) for selected date range in addition to lifetime stats

[AK-Skinner]

As a technical manager, I want to be able to easily gather aggregate summary stats for a given (arbitrary) time period for comparison against other (arbitrary) time periods and baseline data.

When reviewing a check for a given period it is currently not easy to determine if there has been a small change where there is volatility in the check results, at the moment it's a matter of eyeballing the check charts or selecting different periods in reporting and approximating.

For example, when 3rd party service has made a change I want to compare the 24hrs preceding the moment of change to the later 24hrs period up to 12:30 to see if there's been an increase in the mean, P95 and P99 response time values taking into account normal traffic. I'd like to do the same for the early morning / low traffic hours to check for a baseline change and any other periods of interest.

Describe the solution you'd like I would like either or both;

1. The check detail chart interface to include a second 'Summary' stats block for the currently selected timeframe to allow easy comparison to the lifetime stats.
2. The reporting view to provide a similar comparison display, either on the tooltip or as part of the chart. Perhaps the option to select a comparison or 'baseline' period that shows as a separate series alongside the existing series?

Describe alternatives you've considered Alternatively, a more granular export function would allow me to perform these comparisons and analysis myself from clean checkly data. Currently I will do this by exporting from application logs and comparing, but this does bring other factors that can make direct comparisons somewhat more difficult (eg internal timings will not accurately include latency etc). This becomes an obstacle in sharing results to the wider (less technical) stakeholders since qualifiers require explanation.

[tnolet]

@AK-Skinner thanks for reporting. You are completely correct. I want the same! We are completely replacing our metrics backend so we can give all users arbitrary date ranges on any check results view, whether in the individual check view or on the reporting tab.

Currently, we limit this to preselected ranges as we cannot guarantee a palatable response time. So hold on for our new data backend (Clickhouse if you are curious) and we will update the screens to have use selectable ranges.

A comparison display is a very cool idea that I will look at!

[tnolet]

@AK-Skinner we updated our reporting page recently. You can now select arbitrary date ranges. There is not comparison feature (yet). Hope this already helps