airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Python debugger stucks when the first python_callable from a class has **context parameter

Open opeida opened this issue 6 months ago • 0 comments

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Python debugger in VSCode starts infinite variable loading when the first python_callable handled by PythonOperator has the **context parameter. The problem was absent on Airflow 2.10.5.

P.S. Provided DAG is intentionally simplified. The problem affects our workflow in a production environment.

Case 1:

  • 1st callable doesn't have the **context parameter.
  • 2nd callable has the **context parameter.
  • All works fine.

Image

Case 2:

  • 1st callable has the **context parameter.
  • Debugger starts infinite variable loading and reluctantly responds to the Stop command.

Image

What you think should happen instead?

Python debugger should show variables and works properly.

How to reproduce

VSCode's launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Current DAG",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true
        },
    ]
}

test.py

from airflow import DAG
from airflow.providers.standard.operators.python import PythonOperator


class TestClass:

    def func_with_context(self, **context) -> None:
        return None

    def func_without_context(self) -> None:
        return None


with DAG("test_dag") as dag:

    run_func_with_context = PythonOperator(
        task_id="run_func_with_context",
        python_callable=TestClass().func_with_context
    )

    run_func_without_context = PythonOperator(
        task_id="run_func_without_context",
        python_callable=TestClass().func_without_context
    )

    # Leave only 1 line uncommented at a time
    # run_func_with_context >> run_func_without_context
    run_func_without_context >> run_func_with_context

if __name__ == "__main__":
    dag.test()

Operating System

Debian GNU/Linux 12 (bookworm)

Versions of Apache Airflow Providers

apache-airflow-providers-celery==3.11.0 apache-airflow-providers-common-compat==1.7.0 apache-airflow-providers-common-io==1.6.0 apache-airflow-providers-common-sql==1.27.1 apache-airflow-providers-fab==2.2.0 apache-airflow-providers-google==15.1.0 apache-airflow-providers-http==5.3.0 apache-airflow-providers-postgres==6.2.0 apache-airflow-providers-redis==4.1.0 apache-airflow-providers-smtp==2.1.0 apache-airflow-providers-standard==1.2.0

Deployment

Other Docker-based deployment

Deployment details

Extended image based on apache/airflow:slim-3.0.2-python3.12

Anything else?

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

opeida avatar Jun 17 '25 21:06 opeida