Investigate and borrow from robinhood/airflow-prometheus-exporter

[elephantum]

There's a fork with significant changes, seems to be worth a deeper look.

https://github.com/robinhood/airflow-prometheus-exporter

[sdanbury]

@elephantum is it clear why there was a separate fork, rather than just merging the extra metrics into this project?

[dszakallas]

We need some metrics provided by that fork, but it look like it is abandoned, as important bugfixes are left unmerged (https://github.com/robinhood/airflow-prometheus-exporter/pull/25).

Could we add the metrics provided by that fork?

• [ ] airflow_task_duration - I think this is one of the most useful metrics.
• [ ] airflow_task_fail_count - This is fairly useful too.
• [ ] airflow_xcom_param

Scheduler metrics (require a DAG with a specified name)

• [ ] airflow_dag_scheduler_delay
• [ ] airflow_task_scheduler_delay
• [ ] airflow_num_queued_tasks