airflow-dbt-python icon indicating copy to clipboard operation
airflow-dbt-python copied to clipboard

The operator wouldn't recognize profiles_dir if it finds profile.yml in the project_dir

Open utrivedi15five opened this issue 1 year ago • 1 comments

As per the title, the profiles.yml is directly loaded from the project_dir if found and it ignores the profiles_dir. I have different target name in profiles.yml that live under /User/username/.dbt/ and I get error saying the target is not in the list, with list of targets from profiles.yml in /path/to/my/dbt/project/

 DbtRunOperator(
        task_id="dbt_run_hourly",
        project_dir="/path/to/my/dbt/project/",
        profiles_dir="/User/username/.dbt/",
        select=["+tag:hourly"],
        target="development",
        profile="my-project",
        full_refresh=True,
    )

utrivedi15five avatar Apr 01 '24 21:04 utrivedi15five

https://github.com/tomasfarias/airflow-dbt-python/blob/master/airflow_dbt_python/hooks/dbt.py#L400

probably the culprit. It should check if profiles_dir has been specified and if yes then continue reading the profiles_dir

utrivedi15five avatar Apr 01 '24 21:04 utrivedi15five