dbt-core icon indicating copy to clipboard operation
dbt-core copied to clipboard

[CT-813] [Feature] Improve resource-path nomenclature to allow basic regex patterns

Open adamcunnington-mlg opened this issue 2 years ago • 0 comments

Is this your first time opening an issue?

Describe the Feature

It would be really cool if the resource-path was more sophisticated. Currently, a resource-path can be a project name or a model name, and you can nest model names underneath sub-directories. I have a really robust folder structure and want to use this to power functionality, dynamically.

An example of what I want to be able to do is automatically grant permissions to x group where x is a sub-directory. I propose positional capturing groups from a basic regex flavour.

I think this example is fairly self-explanatory of what I want to be able to do:

models:
  my-project:
    some-parent-dir.some-child-dir.(.+).(.+).sql:
        post_hook: GRANT SOME_PERMISSION TO [email protected] ON $2

The GRANT SQL is pseudo code but what I am trying to do here is dynamically grant some access to the table/view created by a model where the directory containing the sql file is the first captured group (some value that resolves to a valid group name when $1 is substituted) and where the second captured group is the name of the created table/view itself (probably a better way of doing this).

There's a lot of room for design compromise here though so mostly looking to spark a discussion around making resource-paths more flexible. The docs already implies that if you don't want something to apply to all children, your only option is to go for an explicit path which is a shame when for example, you have 1 exception to a rule that would otherwise be ubiquitous - or the use case I showed above (dynamic information).

Describe alternatives you've considered

Maintaining a copy and paste of same information. Undermines DRY (as well as general maintainability).

Who will this benefit?

Everyone who wants to use a more advanced syntax for resource-path, without negatively affecting anyone that does not.

Are you interested in contributing this feature?

Perhaps - but is a question of design first

Anything else?

No response

adamcunnington-mlg avatar Jul 06 '22 14:07 adamcunnington-mlg