dagster
dagster copied to clipboard
[DS][36/n] Create ScheduledSince condition
Summary & Motivation
creates a condition which determines if an asset partition has been requested within some lookback period.
It does this by keeping track of a set of (essentially) tuples of (AssetSlice, last_requested_timestamp). These tuples are evicted from the list whenever they age out of the lookback window.
It uses asset_subsets_with_metadata instead of the more generic extra_state field because asset_subsets_with_metadata get rendered in the UI. this means that users will be able to see exactly when an asset partition was last requested (if it was requested within the given lookback period).
How I Tested These Changes
-
#21788
-
#21741
-
#21740
-
#21739
-
#21712
👈
-
#21793
-
#21704
-
#21703
-
#21677
-
#21737
-
#21671
: 1 other dependent PR (#21705
)
-
#21641
: 1 other dependent PR (#21670
)
-
#21640
-
#21648
-
#21615
-
#21613
-
#21612
-
#21573
-
#21546
-
#21545
-
#21541
-
#21540
-
#21539
-
#21538
-
#21537
-
#21536
-
#21535
-
#21521
-
#21520
-
#21511
: 1 other dependent PR (#21512
)
-
#21510
-
#21508
-
#21507
-
#21505
-
#21504
-
#21503
-
#21502
-
#21501
-
#21500
-
#21499
-
master
This stack of pull requests is managed by Graphite. Learn more about stacking.
Join @OwenKephart and the rest of your teammates on Graphite