airflow-dbt-python
airflow-dbt-python copied to clipboard
The operator wouldn't recognize profiles_dir if it finds profile.yml in the project_dir
As per the title, the profiles.yml is directly loaded from the project_dir if found and it ignores the profiles_dir. I have different target name in profiles.yml that live under /User/username/.dbt/ and I get error saying the target is not in the list, with list of targets from profiles.yml in /path/to/my/dbt/project/
DbtRunOperator(
task_id="dbt_run_hourly",
project_dir="/path/to/my/dbt/project/",
profiles_dir="/User/username/.dbt/",
select=["+tag:hourly"],
target="development",
profile="my-project",
full_refresh=True,
)
https://github.com/tomasfarias/airflow-dbt-python/blob/master/airflow_dbt_python/hooks/dbt.py#L400
probably the culprit. It should check if profiles_dir has been specified and if yes then continue reading the profiles_dir