kedro-plugins icon indicating copy to clipboard operation
kedro-plugins copied to clipboard

Published 20 hours ago • verified publisher icon kedro-org

kedro-airflow: implement @task and @dag decorators in DAG, generated by plugin

Open DimedS opened this issue 1 year ago • 2 comments

Description

Following the investigation done in #25:

We will implement the @task and @dag decorators from the TaskFlow API into the DAG generated by the kedro-airflow plugin from the Kedro project. This implementation will:

  1. Make the DAG code shorter and cleaner.
  2. Allow us to move away from using KedroOperator, simplifying the user experience and addressing Kedro plugins issue #482.
  3. Demonstrate our commitment to keeping up with current best practices and open the way for potentially more extensive use of TaskFlow in the future.

Possible Implementation

Modify airflow_dag_template.j2 accordingly

DimedS avatar May 29 '24 12:05 DimedS

This seems contradicts to the conclusion of https://github.com/kedro-org/kedro-plugins/issues/25?

noklam avatar May 29 '24 16:05 noklam

@noklam We discussed a bit more and decided to adopt certain aspects of the TaskFlow Api for now.

ankatiyar avatar May 30 '24 16:05 ankatiyar