airflow-dbt-python
airflow-dbt-python copied to clipboard
Partial Parse is being ignored 😢
Im getting an issue like this
[2024-07-02, 15:26:13 CST] {eventmgr.py:59} INFO - [0m21:26:13.660583 [info ] [MainThread]: Unable to do partial parsing because config vars, config profile, or config target have changed
Usually in my local env it doesn't fail, but in my astronomer deployment it randomly appears. Usually some tasks work properly but then it starts parsing for each task which is not ideal. Important to note that I'm using fs, so I had to add the fs_default connection
My versions are this
airflow==2.8.0 dbt-core==1.4.8 dbt-databricks==1.4.3 dbt-spark==1.4.3 airflow-dbt-python==2.0.0
This is not actually a problem with this package. This log message means exactly what it says "config vars, config profile, or config target have changed" since compile time.
You can get around this with a monkey patch if you are sure that none of these variables should affect the behavior of your models.
from dbt.parser.manifest import ManifestLoader
def is_partial_parsable_always_true(_self: ManifestLoader, _: Any) -> tuple[bool, str | None]:
return True, None
ManifestLoader.is_partial_parsable = is_partial_parsable_always_true