dbt-sugar icon indicating copy to clipboard operation
dbt-sugar copied to clipboard

[regression] properly support more than 1 dbt project under scope

Open bastienboutonnet opened this issue 4 years ago • 0 comments

#Rationale Originally we wanted to provide support for more than one dbt project under a given syrup config. However, this would slow our initial development but it is a regression as we strongly believe dbt-sugar was built with the main objective in mind to keep column definitions consistent across all of a user's tables.

We discussed a few points in the following issues:

  • https://github.com/bitpicky/dbt-sugar/issues/38
  • https://github.com/bitpicky/dbt-sugar/issues/13

But we made a conscious choice in the code to only support one dbt project for now to allow us to have an MVP out sooner (https://github.com/bitpicky/dbt-sugar/blob/38ba2aff80514c8eacebb4d2ee64a7dd38f2acc1/dbt_sugar/core/main.py#L124-L128) --well actually we are sketchy about it now and we should correct that and throw a proper error.

Challenge/Potential Solutions

Achieving this means that we need to be able to represent and store the models information that we parse from the schema.yml files in a one-level more nested dict that also contains which dbt_project it's in so that we can target and use the right profiles (i.e. credentials and db info) to use when we query information schemas.

bastienboutonnet avatar Feb 20 '21 11:02 bastienboutonnet