Export metrics

[hardbyte]

Thinking something similar to how the redis helm chart allows loading a "sidecar" prometheus exporter of metrics. This optional component would export regular updates into our monitoring solution of choice e.g. postgresdb or elasticsearch.

https://github.com/kubernetes/charts/tree/master/stable/redis#configuration

[hardbyte]

Possible prometheus metrics with types:

• number_of_mappings counter
• mappings_count{status="progress"} gauge
• mappings_count{status="failed"} gauge
• matching_rate gauge (e.g. 22931985)
• matching_job_last_success_duration_seconds gauge
• matching_job_last_success_duration_seconds gauge
• matching_job_last_duration gauge
• matching_job_last_size gauge
• matching_job_last_size gauge
• matching_job_last_overlap gauge (e.g. 0.8)
• http_requests_total

We should be able to easily enable metrics export for celery - https://github.com/zerok/celery-prometheus-exporter

We could also do nginx: https://github.com/knyar/nginx-lua-prometheus