dagster icon indicating copy to clipboard operation
dagster copied to clipboard

[DS][39/n] Use FailedSchedulingCondition and ScheduledSince condition in eager policy

Open OwenKephart opened this issue 9 months ago • 1 comments

Summary & Motivation

As title. This was alluded to in the initial eager policy PR, this is putting them in action. In the case that the latest run for an asset partition failed, we will not request the asset if we've already requested it within the last hour. This puts a natural "rate limit" on the asset in the case that it is repeatedly failing, but does not interfere if the asset is materializing as expected.

How I Tested These Changes

OwenKephart avatar May 09 '24 00:05 OwenKephart

  • #21788 Graphite
  • #21741 Graphite 👈
  • #21740 Graphite
  • #21739 Graphite
  • #21712 Graphite
  • #21793 Graphite
  • #21704 Graphite
  • #21703 Graphite
  • #21677 Graphite
  • #21737 Graphite
  • #21671 Graphite: 1 other dependent PR (#21705 Graphite)
  • #21641 Graphite: 1 other dependent PR (#21670 Graphite)
  • #21640 Graphite
  • #21648 Graphite
  • #21615 Graphite
  • #21613 Graphite
  • #21612 Graphite
  • #21573 Graphite
  • #21546 Graphite
  • #21545 Graphite
  • #21541 Graphite
  • #21540 Graphite
  • #21539 Graphite
  • #21538 Graphite
  • #21537 Graphite
  • #21536 Graphite
  • #21535 Graphite
  • #21521 Graphite
  • #21520 Graphite
  • #21511 Graphite: 1 other dependent PR (#21512 Graphite)
  • #21510 Graphite
  • #21508 Graphite
  • #21507 Graphite
  • #21505 Graphite
  • #21504 Graphite
  • #21503 Graphite
  • #21502 Graphite
  • #21501 Graphite
  • #21500 Graphite
  • #21499 Graphite
  • master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @OwenKephart and the rest of your teammates on Graphite Graphite

OwenKephart avatar May 09 '24 00:05 OwenKephart

Merge activity

  • May 14, 10:38 AM EDT: @OwenKephart started a stack merge that includes this pull request via Graphite.
  • May 14, 11:14 AM EDT: Graphite rebased this pull request as part of a merge.
  • May 14, 11:15 AM EDT: @OwenKephart merged this pull request with Graphite.

OwenKephart avatar May 14 '24 14:05 OwenKephart