airflow icon indicating copy to clipboard operation
airflow copied to clipboard

'None' execution date in a few DAG runs across several others

Open ariansacct opened this issue 1 year ago • 2 comments

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.7.3

What happened?

A series of DAG runs were scheduled. Total number is around 30 (set using start_date and end_date params, one for each day of a month). On two of the runs, the value of context['execution_date'] is None, failing the SQL task that uses it. The other runs are fine. All the runs use the same code and provide_context = True in all.

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'schedule_interval': '@daily',
    'provide_context': True,
    "retry_delay": duration(minutes=5),
    'retries': 5
}

What you think should happen instead?

The date should not be null.

How to reproduce

Not fully sure.

Operating System

Ubuntu 20.04.6 LTS

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

It happened a few times. It does not happen with every single DAG source code.

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

ariansacct avatar Feb 21 '24 14:02 ariansacct

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

boring-cyborg[bot] avatar Feb 21 '24 14:02 boring-cyborg[bot]

Can you provide a replica of your DAG. That being said, 'schedule_interval': '@daily', is a DAG level argument. It shouldn't be in the default_args. default_args are arguments you pass to all the tasks in your DAG

ephraimbuddy avatar Feb 21 '24 16:02 ephraimbuddy

Could you mention what you mean by "replica" of the DAG? I'm happy to share the source code (with parts taken out).

ariansacct avatar Feb 24 '24 10:02 ariansacct

I'm happy to share the source code (with parts taken out).

That's what replica means.

potiuk avatar Feb 24 '24 11:02 potiuk

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Mar 10 '24 00:03 github-actions[bot]

This issue has been closed because it has not received response from the issue author.

github-actions[bot] avatar Mar 17 '24 00:03 github-actions[bot]