Allow Dag Processor Manager to log to stdout

[plmnguyen]

Description

On Airflow 2.8.1, the DAG processor manager no longer logs out to stdout (whereas it did on 2.6.2).

Use case/motivation

We currently have a configuration option for the DAG processor to log to stdout via AIRFLOW__LOGGING__DAG_PROCESSOR_LOG_TARGET. A similar configuration AIRFLOW__LOGGING__DAG_PROCESSOR_MANAGER_LOG_TARGET could achieve that.

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[Owen-CH-Leung]

afaik, the DAG processor manager runs in a spawned process and by default output its log to a destinated file defined by dag_processor_manager_log_location. I think this behaviour also applies to v2.6.2

I suggest adding an extra variable in the config like dag_processor_manager_log_stdout , which if being set to True (default is False) , extend the log handler to also include logging to stdout. That way, the log from the spawned process can also be directed to the stdout of airflow scheduler like below :

thoughts ?

[potiuk]

thoughts ?

This is a valid option if someone wants it, all it needs is someone to PR it.

[Owen-CH-Leung]

thoughts ?

This is a valid option if someone wants it, all it needs is someone to PR it.

Ok I'm on it