Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching
closes #39368
By reusing the get_validators function, we can incorporate pattern search to include allow/block lists.
@pankajkoti, Yes, I have tested them. Allow/Block list without pattern worked perfectly. However, I noticed the following behaviour when using pattern matching, which I am not sure if it's right.
With pattern matching enabled:
AIRFLOW__METRICS__METRICS_ALLOW_LIST=.*queued_duration.*: Worked perfectly
AIRFLOW__METRICS__METRICS_ALLOW_LIST=.*get_astronauts.*: Not sure if this is correct
In the 2nd screenshot, I still see several metrics that don't have get_astronauts anywhere in their metrics names. Is this correct behaviour? One of them is airflow.ti.finish
Block lists seem to be working as intended.
AIRFLOW__METRICS__METRICS_BLOCK_LIST=.*duration.*: works perfectly
FYI: I have been using different tags for different settings to differentiate between searches.
Env variables set:
| AIRFLOW__METRICS__STATSD_DATADOG_ENABLED | true |
|---|---|
| AIRFLOW__METRICS__STATSD_DATADOG_TAGS | testNo:C2 |
| DATADOG_API_KEY | ● ● ● ● ● ● ● |
| AIRFLOW__METRICS__METRICS_USE_PATTERN_MATCH | true |
| AIRFLOW__METRICS__METRICS_ALLOW_LIST | .*print_astronaut_craft.* |
Code change LGTM, but I'm not sure that second allow-list example (the one you seemed unsure about) is working as expected. Either way, that is not in scope for this PR if that is broken, but you may want to look into that more if you have the time.
I will look into it. I will create an issue( #39400) and post my findings there.