Make the SerializedDAG & SerializedBaseOp be independent of Task SDK's DAG & BaseOP
Currently, models.BaseOperator and models.DAG use the Task SDK classes.
This was done to maintain backwards-compat and to get 3.0 in a timely manner.
However, we don't want to depend on Task SDK in the Scheduler or the API-server (airflow-core package) - at least not a required dependency.
We only need to ensure we have enough information for the Scheduler & API-server for a DAG & the Operator. This is done via the SerializedDAG & SerializedBaseOp.
So this task involves: figuring out how we can consolidate: SerializedDAG, models.DAG by not inheriting Task SDK; and same for SerializedBaseOp and models.BaseOperator
To do (not ordered by dependency):
- [x] Timetables
- [ ] Assets
- [ ] DAGNode
- [x] NOTSET and ArgNotSet
- [ ] DeadlineAlert (and related things?) - Currently WIP
- [x] BaseOperatorLink
- [x] ExpandInput
- [ ] Partitions
- [ ] Clean up remaining SDK references in Core
- [ ] Take
airflow.serialization.serialized_objectsapart - [ ] Move MappedOperator to
airflow.serialization.definitions
- [x] BaseOperator
- [x] MappedOperator
- [x] DAG
- [x] Param
- [x] ParamsDict
- [x] TaskGroup
Not done
@uranusjr we should also add an issue for BaseTrigger/BaseEventTrigger?
We should eventually, but I’m not sure what to do with triggerer things yet. Let’s put them off for now until we figure triggerer and dag processor split out.