airflow
airflow copied to clipboard
apache
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
BigQueryInsertJobOperator already support OpenLineage for QUERY type jobs, but it lacks Column Level Lineage (CLL). This PR introduces CLL (Column-Level Lineage) to this operator based on SQL parsing, which can...
Wait for #44477 and #44612 to be merged. This PR integrates a newly introduced (#44477) OpenLineage feature into another Dataproc operator. With this enhancement, users no longer need to manually...
Part of https://github.com/apache/airflow/issues/44481 Example DAG used: ```py """ ### DAG Tutorial Documentation This DAG is demonstrating an Extract -> Transform -> Load pipeline """ from __future__ import annotations import json...
This PR integrates a newly introduced (#44477) OpenLineage feature into another Dataproc operator. With this enhancement, users no longer need to manually provide parent job information for the OpenLineage Spark...
### Apache Airflow version 2.10.4 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? I have multiple dag_run of a dag, running parallel on a...
### Official Helm Chart version 1.15.0 (latest released) ### Apache Airflow version 2.9.3 ### Kubernetes Version 1.29.7 ### Helm Chart configuration _No response_ ### Docker Image customizations _No response_ ###...
Co-authored-by: Ulada Zakharava (cherry picked from commit 9186fc57907c89f2f871d54f981f2b6892920e2f) --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow...
### Apache Airflow Provider(s) microsoft-azure ### Versions of Apache Airflow Providers apache-airflow-providers-microsoft-azure==11.1.0 ### Apache Airflow version 2.10.2 ### Operating System linux ### Deployment Google Cloud Composer ### Deployment details _No...
In the FTPSHook it's also possible to pass a buffer (like StringIO/BytesIO) instead of a path, I've refactored the retrieve_file and store_file methods in SFTPHook so those also support those...
It is most likely that we no longer need `gunicornmontor` or `UvicornMonitor` anymore. @ashb 's suggestion is for Airflow `uvicorn.run()` should be enough. Whoever takes this GitHub issue should verify...