dagster
dagster copied to clipboard
dagster-io
Display PartitionMappings information in Dagit
What's the use case?
It would be useful to see the mapping information in the Dagit Asset Catalog (for example, when hovering over the arrows representing assets dependencies) and in the Lineage views.
We can ask the user to provide a description property for his custom PartitionMappings class for this purpose.
This information can also be shown in the structured logs.
Ideas of implementation
No response
Additional information
No response
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
Hi @danielgafni! I see what you're saying on the Asset Catalog side of things, but do you mind elaborating on what you would like to see in the structured logs? Would this be some extra information attached to the input loading event, or something else?
I agree that this would be a nice feature.
We now support fetching partition mappings via graphQL, so we're good to add UI support: https://github.com/dagster-io/dagster/pull/18119
Relevant discussion around UI: https://dagsterlabs.slack.com/archives/C03CCE471E0/p1701226754655879
hey gang - a GraphQL change recently landed that allows us to query partition mappings (a type + basic description) on the dependency lines between assets. I was thinking I’d show these on the asset definition page for starters (and maybe on the graph as well? :flushed:), but I’m realizing it requires more design than I thought. On the details page, it makes sense to show them on or near the upstream/downstream, but we have annotations on both the top and the bottom of the asset already, and it’s not really a property of the asset, just of the relationship between them. It could be confusing that it’s there on some pages and gone on others. So maybe we actually just want a new section on the page listing the mappings? On the graph we’d show them on the arrow itself, which could mean something like we do for dynamic ops (the >* and <* annotations). But I think we’d need a new icon… (current rough draft below, tag can’t really go here because it blocks the kind tags sometimes) (edited)
