dagster icon indicating copy to clipboard operation
dagster copied to clipboard

[DS][36/n] Create ScheduledSince condition

Open OwenKephart opened this issue 9 months ago • 1 comments

Summary & Motivation

creates a condition which determines if an asset partition has been requested within some lookback period.

It does this by keeping track of a set of (essentially) tuples of (AssetSlice, last_requested_timestamp). These tuples are evicted from the list whenever they age out of the lookback window.

It uses asset_subsets_with_metadata instead of the more generic extra_state field because asset_subsets_with_metadata get rendered in the UI. this means that users will be able to see exactly when an asset partition was last requested (if it was requested within the given lookback period).

How I Tested These Changes

OwenKephart avatar May 07 '24 23:05 OwenKephart

  • #21788 Graphite
  • #21741 Graphite
  • #21740 Graphite
  • #21739 Graphite
  • #21712 Graphite 👈
  • #21793 Graphite
  • #21704 Graphite
  • #21703 Graphite
  • #21677 Graphite
  • #21737 Graphite
  • #21671 Graphite: 1 other dependent PR (#21705 Graphite)
  • #21641 Graphite: 1 other dependent PR (#21670 Graphite)
  • #21640 Graphite
  • #21648 Graphite
  • #21615 Graphite
  • #21613 Graphite
  • #21612 Graphite
  • #21573 Graphite
  • #21546 Graphite
  • #21545 Graphite
  • #21541 Graphite
  • #21540 Graphite
  • #21539 Graphite
  • #21538 Graphite
  • #21537 Graphite
  • #21536 Graphite
  • #21535 Graphite
  • #21521 Graphite
  • #21520 Graphite
  • #21511 Graphite: 1 other dependent PR (#21512 Graphite)
  • #21510 Graphite
  • #21508 Graphite
  • #21507 Graphite
  • #21505 Graphite
  • #21504 Graphite
  • #21503 Graphite
  • #21502 Graphite
  • #21501 Graphite
  • #21500 Graphite
  • #21499 Graphite
  • master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @OwenKephart and the rest of your teammates on Graphite Graphite

OwenKephart avatar May 07 '24 23:05 OwenKephart

Merge activity

  • May 14, 10:38 AM EDT: @OwenKephart started a stack merge that includes this pull request via Graphite.
  • May 14, 11:06 AM EDT: Graphite rebased this pull request as part of a merge.
  • May 14, 11:07 AM EDT: @OwenKephart merged this pull request with Graphite.

OwenKephart avatar May 14 '24 14:05 OwenKephart