oliver-twist icon indicating copy to clipboard operation
oliver-twist copied to clipboard

a rule check specification

Open georgim0 opened this issue 4 years ago • 2 comments

add the following rules: lineage checks:

  • 🆗 Disabled Scripts
  • 🆗 Orphan models
  • 🆗 Rejoined models. Given following model dependency: A -> B -> C. Model C should not depend on A
  • 🆗 Staging scripts referencing a staging script in a different area
  • 🆗 Staging scripts referencing a mart
  • 🆗 Marts directly referencing a source

meta checks on physical models(model that is not ephemeral):

  • 🆗 no owner defined as per(https://docs.getdbt.com/reference/resource-properties/meta/#designate-a-model-owner)
  • no description defined
  • no pii defined

advanced lineage check:

  • detect similar sub graphs which they originate from the same set of sources to figure out if there is a potential code reuse(enforce single source of truth, thinking that it would highlight duplicate code)

georgim0 avatar Dec 22 '20 09:12 georgim0

TODO:

  • think about seeds

georgim0 avatar Dec 22 '20 11:12 georgim0

@georgim0 where are you up to with this?

dorzey avatar Jan 20 '21 15:01 dorzey