dagster
dagster copied to clipboard
explicitly include asset check selection on backfill runs
Summary & Motivation
Fixes https://github.com/dagster-io/dagster/issues/25328.
Currently, when asset backfills launch, all the checks for the selected assets are executed. However, these checks aren't explicitly included on the asset_check_selection
property of the DagsterRun
. A consequence of this is that, when the run is re-executed, the asset checks aren't included in the subset job, which causes the error described in the issue linked above.
This problem points to some deeper mismatched assumptions that would be good to fix as well. I.e. it seems like we should either:
- be stricter about the selections included on runs in general
- be looser about how we construct subsetted jobs when doing re-execution
How I Tested These Changes
Ran through the reproduction steps listed on https://github.com/dagster-io/dagster/issues/25328 and verified that I could successfully re-execute the asset check for the run inside the backfill.
Also added a unit test to verify that the check selection is included inside the run request created by the backfill.
Ideally this would include a larger unit test that does something closer to the manual reproduction described above. I explored a couple routes for this, but alas was not able to find one that worked before my PTO.
Changelog
Fixed a bug that caused DagsterExecutionStepNotFoundError
errors when trying to execute an asset check step of a run launched by a backfill.