dag-factory icon indicating copy to clipboard operation
dag-factory copied to clipboard

Getting rid off long imports

Open rchojn opened this issue 4 years ago • 2 comments

Could we get rid off this long import? airflow.operators.bash_operator.BashOperator

Would be greate to have BashOperator instead

rchojn avatar Sep 07 '20 14:09 rchojn

IMHO, Explicit value makes it easy to find related information.

gigkokman avatar Oct 17 '20 04:10 gigkokman

A workaround for this would be to import the required classes in an utils.py file that aggregates everything you want at the root of the dags folder. Then imports would be only from utils: utils.BashOperator or utils.PythonOperator.

You may also rename the operators or extend their functionalities, like adding more templated arguments, e.g.:

# utils.py
from airflow.operators.bash_operator import BashOperator

class CustomBashOperator(BashOperator):
    template_fields = tuple(list(super().template_fields) + ["additional_templated_field"])
# YAML dag factory dag
example_dag:
    owner: "airflow"
    start_date: 2022-01-01
    retries: 1
    retry_delay_sec: 300
  schedule_interval: "0 1 * * *"
  tasks:
    test:
      operator: utils.BashOperator
      bash_command: "echo 'I came from utils.py'
    test_2:
      operator: utils.CustomBashOperator
      bash_command: "echo 'I came from utils.py and I got extended'

guizsantos avatar Dec 11 '22 21:12 guizsantos