apache
Not time to refresh bundle dags-folder when running `airflow standalone`
Apache Airflow version
3.0.0
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When I startup Airflow using airflow standalone, things work as expected:
dag-processor | [2025-04-28T14:19:55.455+0100] {manager.py:748} INFO -
dag-processor | ================================================================================
dag-processor | DAG File Processing Stats
dag-processor | Bundle File Path PID Current Duration # DAGs # Errors Last Duration Last Run At
dag-processor | ----------- ------------------------------ ----- ------------------ -------- ---------- --------------- -------------
dag-processor | dags-folder jaffle_shop_kubernetes.py 0 0
dag-processor | dags-folder example_virtualenv.py 0 0
dag-processor | dags-folder basic_cosmos_task_group.py 0 0
dag-processor | dags-folder hello.py 0 0
dag-processor | dags-folder example_operators.py 0 0
dag-processor | dags-folder simple_dag_async.py 0 0
dag-processor | dags-folder basic_cosmos_dag.py 0 0
dag-processor | dags-folder example_virtualenv_mini.py 86996 0.05s 0 0
dag-processor | dags-folder example_dataset_triggered.py 86994 0.06s 0 0
dag-processor | dags-folder example_dag_variable_access.py 0 0
dag-processor | ================================================================================
triggerer | [2025-04-28T14:19:55.328+0100] {triggerer_job_runner.py:161} INFO - Starting the triggerer
However, after some time, the DAGs vanish from the UI and the following is logged (the DAGs that now have "errors" are the ones that vanished from the UI:
dag-processor | ================================================================================
dag-processor | DAG File Processing Stats
dag-processor | Bundle File Path PID Current Duration # DAGs # Errors Last Duration Last Run At
dag-processor | ----------- ------------------------------ ----- ------------------ -------- ---------- --------------- -------------------
dag-processor | dags-folder jaffle_shop_kubernetes.py 0 1 1.14s 2025-04-28T13:20:00
dag-processor | dags-folder example_virtualenv.py 0 1 2.91s 2025-04-28T13:19:59
dag-processor | dags-folder basic_cosmos_task_group.py 0 1 1.21s 2025-04-28T13:20:00
dag-processor | dags-folder simple_dag_async.py 0 1 1.32s 2025-04-28T13:19:59
dag-processor | dags-folder basic_cosmos_dag.py 0 1 1.07s 2025-04-28T13:20:01
dag-processor | dags-folder hello.py 1 0 0.05s 2025-04-28T13:20:00
dag-processor | dags-folder example_operators.py 1 0 0.20s 2025-04-28T13:20:01
dag-processor | dags-folder example_virtualenv_mini.py 1 0 1.71s 2025-04-28T13:19:57
dag-processor | dags-folder example_dataset_triggered.py 1 0 0.68s 2025-04-28T13:19:56
dag-processor | dags-folder example_dag_variable_access.py 1 0 1.16s 2025-04-28T13:19:58
dag-processor | ================================================================================
dag-processor | [2025-04-28T14:20:25.746+0100] {manager.py:523} INFO - Not time to refresh bundle dags-folder
If I stop Airflow and run airflow dags list-import-errors, there are no errors::
No data found
If I reserialize airflow dags reserialize the DAGs or if I just restart airflow standalone, these DAGs show up again in the UI - until the bundle reparsing happens and things vanish again, as described at the beginning of the message.
The DAGS that are not showing up are dynamically generated, and it may take some time for them to be generated. In Airflow 2, they would be displayed - and if I wanted to be on the safe side, I could set AIRFLOW__CORE__DAGBAG_IMPORT_TIMEOUT and AIRFLOW__CORE__DAG_FILE_PROCESSOR_TIMEOUT , and I would no longer experience issues. Increasing the value of these configurations did not work in Airflow 3.0.0.
I also attempted to increase:
-
AIRFLOW__DAG_PROCESSOR__MIN_FILE_PROCESS_INTERVAL -
AIRFLOW__DAG_PROCESSOR__BUNDLE_REFRESH_CHECK_INTERVAL -
AIRFLOW__DAG_PROCESSOR__REFRESH_INTERVAL -
AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST
Example of configuration used:
export AIRFLOW__CORE__DAGBAG_IMPORT_TIMEOUT=20000
export AIRFLOW__CORE__DAG_FILE_PROCESSOR_TIMEOUT=20000
export AIRFLOW__DAG_PROCESSOR__DAG_FILE_PROCESSOR_TIMEOUT=20000
export AIRFLOW__DAG_PROCESSOR__REFRESH_INTERVAL=600
export AIRFLOW__DAG_PROCESSOR__MIN_FILE_PROCESS_INTERVAL=20000
export AIRFLOW__DAG_PROCESSOR__BUNDLE_REFRESH_CHECK_INTERVAL=600
export AIRFLOW__DAG_PROCESSOR__STALE_BUNDLE_CLEANUP_AGE_THRESHOLD=20000
export AIRFLOW__DAG_PROCESSOR__STALE_BUNDLE_CLEANUP_INTERVAL=20000
export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
{
"name": "dags-folder",
"classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
"kwargs": {
"refresh_interval": 600}
}
]'
What you think should happen instead?
I believe the Airflow logs should:
- Explain there was a timeout parsing those DAGs, how long they took and what is the current threshold
- Recommend what is the best way to increase the threshold to refresh bundles, to avoid "Not time to refresh bundle dags-folder" error
- It may also make sense to log this type of error as error and not only info, and highlight those somewhere in the Airflow UI
How to reproduce
Clone this repo: https://github.com/astronomer/astronomer-cosmos/
Export PYTHONPATH= <your-path>/astronomer-cosmos/
Follow the steps described in the following README and run airflow standalone from this folder:
https://github.com/astronomer/astronomer-cosmos/tree/main/scripts/airflow3
Operating System
MacOS
Versions of Apache Airflow Providers
No response
Deployment
Other
Deployment details
Local airflow standalone
Anything else?
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
I am also encountering this same issue. After some time all of the DAGs will disappear, report the DAGs all have errors just as in the log message above by @tatiana when running in standalone and there are no errors found when using list-import-errors but airflow dags reserialize will bring them back for a short while before they disappear again.
Has anyone been able to solve the issue? I tried using uv on 3.0.0 version as the webpage recommended, and still had the same issue
I am not able to see the dag code changes made on the airflow dag directory on airflow console.
We also have this problem, and we aren't able to manually refresh the DagBag, too, because of https://github.com/apache/airflow/issues/51816
We can't possibly update to 3.0 until it's not fixed.
@ptoshniasu i will suggest using 2.11.0 version. That works fine for me. I did it in WSL2
I tried the 3.0.0 and had same issues.
By the time I faced this issue, I paired with @dstandish on trying to understand the root cause.
We investigated the logs/dag_processor/ correspondent to the DAGs that were failing and found SIGSEGV, as detailed here.
To be able to effectively do the local development of DAGs locally in a MacOS and sucessfully run airflow standalone, I set the following in my bashprofile (.zshrc):
unset https_proxy
unset all_proxy
unset no_proxy
unset http_proxy
export no_proxy='*'
export PYTHONFAULTHANDLER=true
export AIRFLOW__CORE__LOAD_EXAMPLES=False
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
This has helped me avoiding lots of signal terminations that are happening when using Airflow 3 in MacOS and were not being exposed in the airflow standalone stdout nor in the airflow dags list-import-errors.
Would it make sense to expose the DAG processor logs in airflow dags list-import-errors? Or somewhere else? I believe it would - but I'll leave this open for the community to log a new issue.