airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Improve debug logging

Open rawwar opened this issue 1 year ago • 3 comments

Description

Identify and improve debug logging in core and providers. Debug logs should include all the information used to make the request/invocation, especially when interacting with external systems.

Use case/motivation

It helps identify the data used when interacting with external systems and response data. This can be very helpful in case of failures. Many providers already do this. But, goal of this issue is to identify gaps and fill them

Related issues

Parent Issue: https://github.com/apache/airflow/issues/40975

Are you willing to submit a PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

rawwar avatar Oct 19 '24 03:10 rawwar

@omkar-foss, I'd like to work on this issue. I would like to start with the scheduler

Prab-27 avatar Dec 19 '24 08:12 Prab-27

@Prab-27 , feel free to create issues and refer them to this one. We can use this as an Issue to track progress. I'll start tracking them here once you link the issue.

rawwar avatar Dec 19 '24 08:12 rawwar

Sure thanks for picking it up @Prab-27, feel free to create a new issue for the Scheduler debug logging improvements and link it to this issue (as @rawwar suggested above).

Also in case you'd like to discuss or brainstorm something quickly, you can drop a message on #sig-improving-debugging channel on Airflow community Slack, link here.

omkar-foss avatar Dec 20 '24 16:12 omkar-foss