airflow-dbt-python icon indicating copy to clipboard operation
airflow-dbt-python copied to clipboard

Partial Parse is being ignored 😢

Open jsuarezgg opened this issue 1 year ago • 1 comments

Im getting an issue like this

[2024-07-02, 15:26:13 CST] {eventmgr.py:59} INFO - [0m21:26:13.660583 [info ] [MainThread]: Unable to do partial parsing because config vars, config profile, or config target have changed

Usually in my local env it doesn't fail, but in my astronomer deployment it randomly appears. Usually some tasks work properly but then it starts parsing for each task which is not ideal. Important to note that I'm using fs, so I had to add the fs_default connection

My versions are this

airflow==2.8.0 dbt-core==1.4.8 dbt-databricks==1.4.3 dbt-spark==1.4.3 airflow-dbt-python==2.0.0

jsuarezgg avatar Jul 02 '24 21:07 jsuarezgg

This is not actually a problem with this package. This log message means exactly what it says "config vars, config profile, or config target have changed" since compile time.

You can get around this with a monkey patch if you are sure that none of these variables should affect the behavior of your models.

from dbt.parser.manifest import ManifestLoader

def is_partial_parsable_always_true(_self: ManifestLoader, _: Any) -> tuple[bool, str | None]:
    return True, None

ManifestLoader.is_partial_parsable = is_partial_parsable_always_true

millin avatar Feb 28 '25 13:02 millin