dbt-sugar
dbt-sugar copied to clipboard
[smart tag] automatic tags from user patterns
We want to allow users to provide some mappings to use for automated tagging of columns. For example, they may want to say that "everything that contains 'email' should be tagged as PII".
We probably want to make two convenience methods for ease of use such as contains
and full_match
(under the hood equality test) as well as let more programming oriented users sumbit regex patterns
The entry in the sugar_config.yml
could look something like this:
smart_tags:
- PII:
- contains:
- 'email'
- full_match:
- 'name'
- regex:
- '_name$|name_(?!company)'
NOTE: would we want to maybe offer some sort of --dry-run
option? What I mean by this would be we that we may want to tell users "based on your conditions (especially valuable for people using regexes) we would tag x, y, x as PII"