dagster icon indicating copy to clipboard operation
dagster copied to clipboard

[DS][32/n] Update UpdatedSinceCronCondition

Open OwenKephart opened this issue 9 months ago • 1 comments

Summary & Motivation

First, realized that this wasn't actually tested. Added real tests, and updated the implementation to be slightly more efficient. In particular, previously whenever any partition of a given asset was updated since the previous tick (not near a cron boundary), we'd end up doing a somewhat-inefficient db query to get all the materialization events since the previous cron tick. Instead, we can just use the (probably already-fetched) information about materializations since the previous tick and keep track of that. Not a game changer or anything, but a bit nicer.

How I Tested These Changes

OwenKephart avatar May 06 '24 23:05 OwenKephart

  • #21788 Graphite
  • #21741 Graphite
  • #21740 Graphite
  • #21739 Graphite
  • #21712 Graphite
  • #21793 Graphite
  • #21704 Graphite
  • #21703 Graphite
  • #21677 Graphite 👈
  • #21737 Graphite
  • #21671 Graphite: 1 other dependent PR (#21705 Graphite)
  • #21641 Graphite: 1 other dependent PR (#21670 Graphite)
  • #21640 Graphite
  • #21648 Graphite
  • #21615 Graphite
  • #21613 Graphite
  • #21612 Graphite
  • #21573 Graphite
  • #21546 Graphite
  • #21545 Graphite
  • #21541 Graphite
  • #21540 Graphite
  • #21539 Graphite
  • #21538 Graphite
  • #21537 Graphite
  • #21536 Graphite
  • #21535 Graphite
  • #21521 Graphite
  • #21520 Graphite
  • #21511 Graphite: 1 other dependent PR (#21512 Graphite)
  • #21510 Graphite
  • #21508 Graphite
  • #21507 Graphite
  • #21505 Graphite
  • #21504 Graphite
  • #21503 Graphite
  • #21502 Graphite
  • #21501 Graphite
  • #21500 Graphite
  • #21499 Graphite
  • master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @OwenKephart and the rest of your teammates on Graphite Graphite

OwenKephart avatar May 06 '24 23:05 OwenKephart

Merge activity

  • May 14, 10:38 AM EDT: @OwenKephart started a stack merge that includes this pull request via Graphite.
  • May 14, 10:55 AM EDT: Graphite rebased this pull request as part of a merge.
  • May 14, 10:56 AM EDT: @OwenKephart merged this pull request with Graphite.

OwenKephart avatar May 14 '24 14:05 OwenKephart