dagrules icon indicating copy to clipboard operation
dagrules copied to clipboard

Presence of exposures breaks dagrules with a KeyError when looking up child nodes

Open awshepard opened this issue 2 years ago • 1 comments

Note: this is based on a cursory understanding of the mechanics of the manifest, as well as dagrules. Apologies for any errors/misstatements.

When a project contains exposures, dbt adds them as children in the child_map in manifest.json, but does not create actual nodes for them. When selecting nodes for validation, dagrules references the child map to identify child nodes, then gets additional metadata from the node entry. Since exposures don't exist as nodes, dagrules throws a key error:

Traceback (most recent call last):
  File "/usr/local/bin/dagrules", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/dagrules/cli.py", line 43, in main
    dagrules.core.check(config, manifest)
  File "/usr/local/lib/python3.9/site-packages/dagrules/core.py", line 225, in check
    subjects = rule_subjects(
  File "/usr/local/lib/python3.9/site-packages/dagrules/core.py", line 278, in rule_subjects
    selected_nodes[node]["child_params"] = {
  File "/usr/local/lib/python3.9/site-packages/dagrules/core.py", line 279, in <dictcomp>
    child: flat_nodes[child] for child in selected_nodes[node]["children"]
KeyError: 'exposure.<project>.<exposure name>'
make: *** [dagrules_check] Error 1

I'd be happy to collaborate on and/or contribute a solution (pending digging in a bit deeper), but wanted to get this logged for reference.

awshepard avatar Jul 27 '22 17:07 awshepard

+1, running into the same error

paulfry-payroc avatar Jan 23 '24 15:01 paulfry-payroc