apache
AIP-88: Lazy expandable task mapping
AIP-88 allows triggerers to yield and have the scheduler/process handle multiple events in a single trigger run, enabling true streamable dynamic task mapping for async operators (e.g., HttpOperator/MSGraphAsyncOperator). This reduces repeated context switching and "ping-pong" between scheduler → worker → triggerer, enabling efficient in-trigger pagination and lazy task expansion.
To be able to complete this PR, I first need the the Fix rendering of template fields with start from trigger PR to be merged, once that one is merged I can finish this one as it heavily depends on the start_from_trigger mechanism which was broken and temporarily disabled.
Current implementation:
New improved mechanism:
https://www.youtube.com/watch?v=mWUAk-Gwmws
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
