prefect icon indicating copy to clipboard operation
prefect copied to clipboard

prefect-dbt: from_file method for BigQueryTargetConfigs

Open robfreedy opened this issue 1 year ago • 0 comments

First check

  • [X] I added a descriptive title to this issue.
  • [X] I used the GitHub search to find a similar request and didn't find it.
  • [X] I searched the Prefect documentation for this feature.

Prefect Version

2.x

Describe the current behavior

Currently, you have to define a BigQueryTargetConfigs block with BigQuery configurations manually in python code or in the Prefect UI, which can then get passed to the DbtCliProfile to save the credentials. This usually means a lot of copying and pasting from the dbt profiles.yml or elsewhere.

Describe the proposed behavior

Add a from_file method to the BigQueryTargetConfigs block to automatically populate the credentials and config into a new instance of a BigQueryTargetConfigs block given an existing dbt profiles.yml file.

Example Use

big_query_target_configs = BigQueryTargetConfigs.from_file(file_path="/path/to/profileyaml/.dbt/profile.yml")
dbt_cli_profile = DbtCliProfile(
            name="jaffle_shop",
            target="dev",
            target_configs=big_query_target_configs,
        )

Additional context

No response

robfreedy avatar May 16 '24 13:05 robfreedy