Matt Matravers

This PR describes many of the points raised in https://github.com/datahub-project/datahub/issues/5635

I'm going to re-organise a lot of this document to be clear what designs are viable and which are not.

> Who is Peter? I think that's @szalai1. > If you mean the comment above, what this PR solves... I have removed CodaHale and instead of it, I have added...

> * Any change that changes exposed metrics at this level, needs to be feature flagged, so teams relaying on the metrics for alerting and dashboarding can rollout whenever they...

@djordje-mijatovic I think you'll need to merge changes from master to fix the red builds. Going over this again, I have some suspicion that we might not have an exactly...