dag-factory
dag-factory copied to clipboard
Getting rid off long imports
Could we get rid off this long import?
airflow.operators.bash_operator.BashOperator
Would be greate to have
BashOperator
instead
IMHO, Explicit value makes it easy to find related information.
A workaround for this would be to import the required classes in an utils.py
file that aggregates everything you want at the root of the dags folder. Then imports would be only from utils: utils.BashOperator
or utils.PythonOperator
.
You may also rename the operators or extend their functionalities, like adding more templated arguments, e.g.:
# utils.py
from airflow.operators.bash_operator import BashOperator
class CustomBashOperator(BashOperator):
template_fields = tuple(list(super().template_fields) + ["additional_templated_field"])
# YAML dag factory dag
example_dag:
owner: "airflow"
start_date: 2022-01-01
retries: 1
retry_delay_sec: 300
schedule_interval: "0 1 * * *"
tasks:
test:
operator: utils.BashOperator
bash_command: "echo 'I came from utils.py'
test_2:
operator: utils.CustomBashOperator
bash_command: "echo 'I came from utils.py and I got extended'