dbt-core
dbt-core copied to clipboard
[Bug] state:modified.macros can detect false positives if package macro has same name as builtin (even though it won't be used)
Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
Installing a package that includes a macro with the same name as a builtin macro can trigger false positives for state:modified.macros
— even though that macro will not override the builtin in actuality / at runtime.
Expected Behavior
state:modified.macros
reflects changes to macros that are actually being called at runtime, and fully ignore same-named macros in packages that won't actually ever be called.
Steps To Reproduce
Let's say every model in my project depends on a certain macro. For example, that macro could be:
Let's say that macro calls another builtin macro - run_query
. (This is the case for dbt_snow_mask.apply_masking_policy
, which calls it via apply_masking_policy_list_for_sources
or apply_masking_policy_list_for_models
.)
Let's say you install a new package that has a custom implementation of run_query
:
{% macro run_query(query) %}
{{ exceptions.raise_compiler_error("you aren't supposed to call me") }}
{% endmacro %}
This has no functional effect on your project, because builtin macros can only be overridden if defined within the root project. Even if the builtin run_query
macro were dispatchable (it's not), it wouldn't be actually used until the root project explicitly opts in via dispatch
config.
And yet. As soon as you install that package, and run dbt ls -s state:modified
, you'll see that every single model is selected as "modified." Why? Because dbt seems to think that the post-hook
with open("target/manifest.json") as f:
d = json.load(f)
d['macros']['macro.dbt_snow_mask.apply_masking_policy_list_for_sources']['depends_on']
{'macros': ['macro.dbt_snow_mask.get_meta_objects', 'macro.dbt_utils.get_query_results_as_dict', 'macro.localpackage.run_query']}
Why is 'macro.localpackage.run_query'
showing up there? At runtime, this will actually be macro.dbt.run_query
!
Relevant log output
No response
Environment
- OS: Sonoma 14.4.1
- Python: 3.10.11
- dbt: 1.8.0 / main
Which database adapter are you using with dbt?
No response
Additional Context
No response