dagster icon indicating copy to clipboard operation
dagster copied to clipboard

RFC: `Definitions.get_all_asset_specs`

Open sryza opened this issue 9 months ago • 2 comments

Summary & Motivation

One of the requests that has been coming up in catalog-related user research is the ability to enforce metadata completeness at CI time. E.g. "ensure all my assets have X tag set".

It's possible to do this right now. But it's ugly:

def test_all_assets_have_storage_kind_tag():
    asset_keys_with_missing_storage_kinds = [
        asset_key
        for asset_key, assets_def in defs.get_repository_def().assets_defs_by_key.items()
        if assets_def.tags_by_key.get(asset_key)["dagster/storage_kind"] is None
    ]

    assert asset_keys_with_missing_storage_kinds == []

It would be a lot prettier if you could just loop over asset specs:

def test_all_assets_have_storage_kind_tag():
    asset_keys_with_missing_storage_kinds = [
        asset_spec.key
        for asset_spec in defs.get_all_asset_specs()
        if asset_spec.tags.get("dagster/storage_kind") is None
    ]

    assert asset_keys_with_missing_storage_kinds == []

How I Tested These Changes

sryza avatar May 20 '24 20:05 sryza