dbt-sugar icon indicating copy to clipboard operation
dbt-sugar copied to clipboard

[smart tag] automatic tags from user patterns

Open bastienboutonnet opened this issue 3 years ago • 0 comments

We want to allow users to provide some mappings to use for automated tagging of columns. For example, they may want to say that "everything that contains 'email' should be tagged as PII".

We probably want to make two convenience methods for ease of use such as contains and full_match (under the hood equality test) as well as let more programming oriented users sumbit regex patterns

The entry in the sugar_config.yml could look something like this:

smart_tags:
    - PII:
        - contains:
            - 'email'
        - full_match:
            - 'name'
        - regex:
            - '_name$|name_(?!company)'

NOTE: would we want to maybe offer some sort of --dry-run option? What I mean by this would be we that we may want to tell users "based on your conditions (especially valuable for people using regexes) we would tag x, y, x as PII"

bastienboutonnet avatar Apr 05 '21 15:04 bastienboutonnet