update statsd-exporter mappings

[Maor-Shmueli]

Updating statsd-exporter mappings.yml file, for all current Airflow supported metrics that require mapping - metrics with variable name that can be mapped to Prometheus label (like job/dag/task_id).

We should use both regex and glob matching types cause glob does not support catching labels that do not divide by dot(.) like <job_name>_start.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[boring-cyborg[bot]]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst) Here are some useful points:

• Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] Slack: https://s.apache.org/airflow-slack

[potiuk]

@dstandish @jedcunningham -> WDYT?

[Maor-Shmueli]

@dstandish @jedcunningham Just to clarify, I ran into this file while me and my team set up statsd-exporer for airflow metrics. I noticed that it wasn't maintained for a while and was not aligned with the current version airflow metrics.

So I did some tidying up, and basically:

• I took all metrics that need a mapping (metrics with variable labels) from the documentation, and created a mapping for each one of them.
• The mappings are by the order of the metrics in docs, and I added the original metric name as a comment before the mapping.
• Some of the labels was not aligned with the labels in docs (like operator → operator_name) so I changed them.

[potiuk]

I marked it as candidate to merge for Helm Chart 1.7.0 - pending review and merge by the team :)

[jedcunningham]

Unfortunately, adding mappings like this is a breaking change.

For example, lets look at dag_processing.last_duration.<dag_file>. Right now that's airflow_dag_processing_last_duration_hello_world but will end up being `airflow_dag_processing_last_duration{dag_file="hello-world"}.

Same thing with the operator vs operator_name thing - we can't change it for this release.

So unfortunately, until we are ready for 2.0 of the chart we can't make these changes.

I think https://github.com/apache/airflow/pull/25219 is closer to what we need for now as it opens up the possibility for users of the charts to provide their own mappings.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.