dagster icon indicating copy to clipboard operation
dagster copied to clipboard

missing documentation for attaching metadata to asset materialization

Open dagsir[bot] opened this issue 3 years ago • 1 comments

Dagster Documentation Gap

This issue was generated from the slack conversation at: https://dagster.slack.com/archives/C01U954MEER/p1659959811619559?thread_ts=1659959811.619559&cid=C01U954MEER

Conversation excerpt

U0353PNKHQU: What I mean particularly, is that some of the alternatives don't seem to link up: These assets are partitioned on a static list of values and, as best I can tell from the docs, the only way to set up a partitioned asset is via the partitions_def argument to the @asset decorator. If I want to add metadata to a materialisation, the docs only mention how to do it via an op based asset or from a custom IO manager. So I could make them an op in a graph-backed asset maybe? But I can't see any docs on how to partition graph-backed assets? (I know you can actually because I went digging in the source for AssetsDefinition.from_graph() but it's not at all clear from the docs) U029405KYRY: I know that one way you can log metadata is via the context.add_output_metadata method, which will log whatever is in the dict to the AssetMaterialization event that running the asset produces. I'm not sure how this plays with partitions yet. U029405KYRY: @asset def my_asset(context): context.add_output_metadata({"how_cool": "so_cool"}) return [1, 2, 3] U029405KYRY: Also, my understanding is that all assets are "op-backed" assets. Under the hood, @asset is basically an @op with superpowers U028M11QNDD: Hi Ben! In addition to what Stephen mentioned, you can also add metadata to an asset materialization by doing something like this: yield Output(5, metadata={"my_metadata": 5}) U028M11QNDD: I can file a ticket to document partitioning graph-backed assets U028M11QNDD: <@U018K0G2Y85> docs no documentation for partitioned graph-backed assets U018K0G2Y85: Created issue at: https://github.com/dagster-io/dagster/issues/9281 U028M11QNDD: <@U018K0G2Y85> docs missing documentation for attaching metadata to asset materialization


Message from the maintainers:

Are you looking for the same documentation content? Give it a :thumbsup:. We factor engagement into prioritization.

dagsir[bot] avatar Aug 08 '22 18:08 dagsir[bot]

We should document that metadata can be attached to asset materializations via:

yield Output(5, metadata={"my_metadata": 5})

in this section

clairelin135 avatar Aug 08 '22 19:08 clairelin135