airflow
airflow copied to clipboard
apache
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
…nction this condition will allow us to have the choice of purging expired dags or keeping them --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more...
### Description Create operators and Hook to interact with Databricks' DBFS (https://docs.databricks.com/api/workspace/dbfs) ### Use case/motivation As per latest databricks plugin (https://github.com/apache/airflow/tree/main/airflow/providers/databricks) there is no possibility to interact with DBFS API....
PR of `Use kubeconfig as one of the optional parameters for creating KubernetesHook` closes: https://github.com/apache/airflow/issues/39227 When we use `KubernetesPodOperator`, we need to create `KubernetesHook` in order to connect to Kubernetes...
### Description We are trying to upgrade to `urllib3` 2.x but faced a few challenges in terms of dependencies w.r.t Databricks, in particular we use `dbt-databricks` as well as `apache-airflow-providers-databricks`:...
closes: https://github.com/apache/airflow/issues/34023 Redo of the #34337 Relates to https://github.com/apache/airflow/issues/35541#issuecomment-1826259201 @uranusjr Can you please take a look at the approach? I haven't checked the performance yet, but if the approach is...
Adding a series of hook functions for OpenAI [File](https://platform.openai.com/docs/api-reference/files) and [Vector Store](https://platform.openai.com/docs/api-reference/vector-stores) endpoints.
### Apache Airflow version 2.9.0 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? We have recently upgraded from Airflow 2.4.0 to 2.9.0. The migration...
`status__in` should be used instead https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow Improvement Proposal...
Some refactor cloudpickle support which added into the https://github.com/apache/airflow/pull/38531 Use new `serializer` parameter for specify serialiser, this one for avoid boolean trap. Deprecate `use_dill` parameter in favor of `serializer="dill"` and...
This fixes the deferrable mode for DataflowTemplatedJobStartOperator and DataflowStartFlexTemplateOperator. Previously the deferrable mode was implemented in a way that made most of the task execution run in the sync mode,...
