dagster icon indicating copy to clipboard operation
dagster copied to clipboard

utility to build an asset sensor from a software-defined asset

Open sryza opened this issue 2 years ago • 2 comments

  • By default, it would trigger whenever the upstream assets are updated. Will need to figure out how this works when there are multiple upstream assets - AND or OR?
  • If the assets are partitioned, it would trigger the corresponding partition in the downstream assets when the corresponding partitions in the upstream assets are updated.

Relevant requests:

  • https://dagster.slack.com/archives/C01U5LFUZJS/p1659067359346519

sryza avatar Jul 22 '22 15:07 sryza

Hey!

My 5 cents:

  1. Probably the default should be AND. We can allow providing an optional custom function to evaluate whether the sensor should be triggered or not based on the upstream assets condition, the default function being all.
  2. By "corresponding" you mean "mapped", right?
  3. Maybe this can also be built from an existing job created via define_assets_job

danielgafni avatar Jul 22 '22 18:07 danielgafni

Probably the default should be AND.

That matches my intuition too.

By "corresponding" you mean "mapped", right?

Right.

Maybe this can also be built from an existing job created via define_assets_job

Good idea.

sryza avatar Jul 22 '22 18:07 sryza