Metrics - DagRun Schedule Delay

[ferruzzi]

Apache Airflow version

2.6.2

What happened

The timers airflow_dagrun.<dag_id>.first_task_scheduling_delay and airflow_dagrun.schedule_delay.<dag_id> both get emitted correctly in StatsD but not in OTel.

I am submitting this as an Issue since I will be a little distracted for the next bit and figured someone may be able to have a look in the meantime. Please do not assign it to me, I'll get it when I can is nobody else does.

What you think should happen instead

Behavior should be consistent.

How to reproduce

To reproduce, you can run Breeze with the statsd or the otel integration (for example breeze start-airflow --integration otel) and run the following DAG, then open the OTel or StatsD raw data view to verify.

from airflow import DAG
from airflow.decorators import task
from airflow.utils.timezone import datetime


@task
def task1():
    return 'Hello'


@task
def task2():
    return 'World!'

@task
def task3(in1, in2):
    print(f'{in1} {in2}')


with DAG(
    dag_id='taskflow_demo',
    start_date=datetime(2021, 1, 1),
    schedule=None,
    catchup=False
) as dag:

    task3(task1(), task2())

You will see that OTel does emit airflow_dagrun_first_task_scheduling_delay and airflow_dagrun_schedule_delaywith tags containing the dag_id, but it does not get the version of the matric which contains the embedded dag_id.

This MIGHT be due to the name length restrictions in OTel but if that is the case then it should be emitting a truncated name, not dropping/ignoring it.

Operating System

ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

[potiuk]

not stale