Alerting

[wlach]

This issue covers an initial implementation of alerting in Mission Control. Initially our expectation was that we would set "thresholds" for each measure above/below which we would alert. However, after some thought (e.g. https://wlach.github.io/blog/2017/10/better-or-worse-by-what-measure/) I think a better strategy is to set such thresholds dynamically based on the median results for the previous few releases at the same time as the current one. This is likely to give better results.

It's pretty easy to think about how this would work in the context of a platform/channel combination that releases relatively infrequently (e.g. windows release). It might be slightly more complicated for channels that release frequently (nightly) or platforms where we have a lot of different builds submitting their data (linux). But I'm sure we can figure something out... worst case we can use different aggregation/summarization strategies for different channels and/or platforms.