airflow icon indicating copy to clipboard operation
airflow copied to clipboard

feat: automatically inject OL info into spark properties in DataprocCreateBatchOperator

Open kacpermuda opened this issue 1 year ago • 0 comments

This PR integrates a newly introduced (#44477) OpenLineage feature into another Dataproc operator. With this enhancement, users no longer need to manually provide parent job information for the OpenLineage Spark integration to generate the parent job facet. Detailed information about this feature can be found in the description of #44477.


^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

kacpermuda avatar Dec 03 '24 15:12 kacpermuda