Flask-MonitoringDashboard
Flask-MonitoringDashboard copied to clipboard
Results with very little evidence are also included in reports

This example claims a 53% decrease in latency, but for these two intervals there are only two measurements for this endpoint. Obviously, ridiculous to include a result like this, given such little evidence.
I have two simple suggestions:
- Include all results, but mention the size of the samples used. Add somewhere: "Based on two samples with 1 measurement and 2 measurements". This way people can make their own judgement about the validity of this result.
- Exclude results that compared small samples -> Either sample needs to have at least 30 measurements for example.
These are two simple options that only take a few lines at most to implement. There are probably more sophisticated ways, but I'll research those during my thesis project :)