dagster
dagster copied to clipboard
utility to build an asset sensor from a software-defined asset
- By default, it would trigger whenever the upstream assets are updated. Will need to figure out how this works when there are multiple upstream assets - AND or OR?
- If the assets are partitioned, it would trigger the corresponding partition in the downstream assets when the corresponding partitions in the upstream assets are updated.
Relevant requests:
- https://dagster.slack.com/archives/C01U5LFUZJS/p1659067359346519
Hey!
My 5 cents:
- Probably the default should be AND. We can allow providing an optional custom function to evaluate whether the sensor should be triggered or not based on the upstream assets condition, the default function being
all
. - By "corresponding" you mean "mapped", right?
- Maybe this can also be built from an existing job created via
define_assets_job
Probably the default should be AND.
That matches my intuition too.
By "corresponding" you mean "mapped", right?
Right.
Maybe this can also be built from an existing job created via define_assets_job
Good idea.