apache
Add helpful debug statements to Google Cloud Dataproc hook
Apache Airflow Provider(s)
Versions of Apache Airflow Providers
No response
Apache Airflow version
airflow-2.9.1
Operating System
NA
Deployment
Google Cloud Composer
Deployment details
No response
What happened
The various hooks in the Google Cloud Dataproc hook do not have debug statements that would make it easier for developers to identify and resolve issues related to passing incorrect field values to the APIs.
One specific example is the create_batch, method. By including a debug statement that prints the batch object, developers can quickly verify the object's contents and identify any discrepancies between the fields and the Dataproc API.
What you think should happen instead
No response
How to reproduce
Submit any DAG using Google Cloud Batch Operator
Anything else
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
If you want - feel free to contribute - otherwise it will have to wait for someone to pick it up- marked as good first issue.
If you want - feel free to contribute - otherwise it will have to wait for someone to pick it up- marked as good first issue.
I would like to pick it up. Can you please tell me whom to contact if I need some help while adding statements.
