airflow
airflow copied to clipboard
Have SLA docs reflect reality
What do you see as an issue?
The SLA documentation currently states the following:
An SLA, or a Service Level Agreement, is an expectation for the maximum time a Task should take. If a task takes longer than this to run...
However this is not how SLAs currently work in Airflow, the SLA time is calculated from the start of the DAG not from the start of the task.
For example if you have a DAG like this the SLA will always trigger after the DAG has started for 5 minutes even though the task never takes 5 minutes to run:
import datetime
from airflow import DAG
from airflow.sensors.time_sensor import TimeSensor
from airflow.operators.python import PythonOperator
with DAG(dag_id="my_dag", schedule_interval="0 0 * * *") as dag:
wait_time_mins = TimeSensor(target_time=datetime.time(minute=10))
run_fast = PythonOperator(
python_callable=lambda *a, **kw: True,
sla=datetime.timedelta(minutes=5),
)
run_fast.set_upstream(wait_time_mins)
Solving the problem
Update the docs to explain how SLAs work in reality.
Anything else
No response
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Feel free to update the docs.
Feel free to update the docs.
Will do, made issue mostly to see if someone would disagree with me. I'll make a PR to update the docs soon.
assigned you :)