dagster icon indicating copy to clipboard operation
dagster copied to clipboard

[AssetCondition] Static constructors for AssetConditions

Open OwenKephart opened this issue 1 year ago • 1 comments

Summary & Motivation

Opens up a subset of the existing AutoMaterializeRules to being constructed directly as AssetConditions.

Note a small change in the definition of the parent_newer rule, as it was designed under the assumption that its candidate_subset would always be all asset partitions of the asset. This is no longer the case, so I've slightly updated the implementation.

In essence, it works by tracking incremental changes in the state of the asset between ticks, and so if it ignores updates to a certain partition on a tick because that partition is not in the candidate set, then it will never know about that update. Instead, we let it track updates across all asset partitions even if they are not in the candidate subset.

How I Tested These Changes

OwenKephart avatar Feb 16 '24 21:02 OwenKephart

Current dependencies on/for this PR:

  • #19875 Graphite 👈
  • master

This stack of pull requests is managed by Graphite.

OwenKephart avatar Feb 16 '24 21:02 OwenKephart