Add TTL-enabled LRU cache for StatsD metrics aggregation

[shubham36deshpande]

…Helm chart

This PR introduces a TTL (time-to-live) mechanism in conjunction with an LRU (least-recently-used) cache for StatsD metric aggregation. The change is designed to automatically clean up stale or unused metric entries, preventing uncontrolled memory growth in long-running statsd daemons—a problem highlighted in issue #50645.

closes: #50645

---

^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[boring-cyborg[bot]]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] Slack: https://s.apache.org/airflow-slack

[jscheffl]

Some static checks fail....

though I am not an statsd expert... @AutomationDev85 Do you have an opinion to the configs?

[eladkal]

tests are failing

[jscheffl]

@shubham36deshpande can you add a newsfragment as Jed recommended in like the other chart/newsfragments/53350.significant.rst?

[shubham36deshpande]

We should also add a newsfragment describing the new values and explaining how to get the old behavior back.

I have added the newsfragment for these cache related changes as 51792.significant.rst

[shubham36deshpande]

@jedcunningham , can I please get your approval here?

[jscheffl]

I see after some pushes over and over failures in CI. Can you please check and read https://github.com/apache/airflow/blob/main/contributing-docs/03a_contributors_quick_start_beginners.rst to set-up your local development environment?

[shubham36deshpande]

@jscheffl , Yes I did that before raising the PR, but let me do it again before merging any changes further! Thanks for the help!