Macro databricks__get_columns_from_information_schema is not working as expected

[HantonMing]

Describe the bug Unity catalog is disabled for us and we have trouble with dbt run -s elementary. In the macro elementary/macros/edr/metadata_collection/get_columns_from_information_schema.sql, another macor is defined for databricks ,which is databricks__get_columns_from_information_schema. The if condition in this macro,

{% if target.catalog is none %}
        {# Information schema is only available when using Unity Catalog. #}
        {% do return(elementary.get_empty_columns_from_information_schema_table()) %}
    {% endif %}

is not working properly in that if condition target.catalog is none is never met. In other words, return(elementary.get_empty_columns_from_information_schema_table()) is never executed. I know this because I made changes to rest of the macro, and changes were reflected in compiled query. To Reproduce Steps to reproduce the behavior:

1. Install elementary-data following instructions
2. Run dbt run -s elementary
3. Error message "Unity Catalog is not abled on this cluster"

Expected behavior Disabled Unity Catalog should be correctly detected. And workaround macro is run.

Screenshots

Environment (please complete the following information):

• edr Version: [0.14.0]
• dbt core version: [1.7.8]
• dbt databricks version: [1.7.7]

Additional context Add any other context about the problem here.

[haritamar]

Hi @HantonMing , Thank you for opening this issue and sorry for such a long delay in our reply.

I'm not sure if this is still relevant to you, but actually would love to know - you mentioned the condition target.catalog is none is never met. Do you manage to see what value does it contain? Does it contain a legitimate value even though you are not using Unity Catalog?

[hamzamazhar]

@haritamar Thanks a lot for getting back to me. To answer your question, in my case, the target.catalog is hive_metastore. As mentioned in my original ticket, I am not specifying the field catalog in the profiles.yaml or anywhere else in the project.

[haritamar]

@hamzamazhar - thanks, that's actually very helpful, so it's a pretty simple fix.

[haritamar]

Hi all - We have a PR for the bugfix to the macro - https://github.com/elementary-data/dbt-data-reliability/pull/722/files

That being said, the information_schema_columns model was actually removed in the recent versions of the package - so practically this issue is already resolved.

So I'm going ahead and closing this issue - please upgrade to the new version! (and feel free to re-open if you feel the issue persists even after upgrading, it shouldn't)