Elementary-data 0.18.3 report errors when using Databricks

[balexander]

Describe the bug A fresh install of elementary-data[databricks]==0.18.3 fails when running edr report. The error message is below. After failure, the report is generated but shows a "No Test Results" message.

The failure seems to be because elementary-data[databricks]==0.18.3 installed the latest version of dbt-databricks, version 1.10.3. The problem is fixed by manually downgrading dbt-databricks to version 1.10.1.

Error Message:

ERROR:elementary.monitor.data_monitoring.report.data_monitoring_report:Could not generate the report - Error: Failed to run dbt command.
{"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  'Macro' object has no attribute 'config'", "level": "error", "invocationId": "2e9f2a82-02cf-4190-9002-1b6ab4f033b0", "pid": 2143, "thread": "MainThread", "ts": "2025-06-11T16:21:13.556338Z"}, "data": {"exc": "Database Error\n  'Macro' object has no attribute 'config'"}}
Please reach out to our community for help with this issue.
Traceback (most recent call last):
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/report/report.py", line 84, in get_report_data
    tests_api = TestsAPI(
                ^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 52, in __init__
    self.test_results_db_rows = self._get_test_results_db_rows(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 64, in _get_test_results_db_rows
    return self.tests_fetcher.get_all_test_results_db_rows(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/fetchers/tests/tests.py", line 26, in get_all_test_results_db_rows
    run_operation_response = self.dbt_runner.run_operation(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/command_line_dbt_runner.py", line 177, in run_operation
    result = self._run_command(
             ^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/command_line_dbt_runner.py", line 114, in _run_command
    result = self._inner_run_command(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/api_dbt_runner.py", line 54, in _inner_run_command
    raise DbtCommandError(
elementary.exceptions.exceptions.DbtCommandError: Failed to run dbt command.
{"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  'Macro' object has no attribute 'config'", "level": "error", "invocationId": "2e9f2a82-02cf-4190-9002-1b6ab4f033b0", "pid": 2143, "thread": "MainThread", "ts": "2025-06-11T16:21:13.556338Z"}, "data": {"exc": "Database Error\n  'Macro' object has no attribute 'config'"}}

To Reproduce Steps to reproduce the behavior:

1. pip install "elementary-data[databricks]
2. run edr report <- the failure happens here

Expected behavior The report should be built successfully.

Environment (please complete the following information):

• Elementary CLI (edr) version: 0.18.3
• Elementary dbt package version: 0.18.3
• dbt version you're using: 1.9.8
• Data warehouse: Databricks
• Infrastructure details: Running the elementary cli locally on a mac. Dbt is hosted in dbt cloud.

Additional context None

Would you be willing to contribute a fix for this issue? Yes.

[Jeremynadalmirakl]

I just had the same issue. It is also there using dbt-databricks==1.10.2.

[arbiv]

@balexander, @Jeremynadalmirakl thanks for reporting this issue! We're looking into it

[arbiv]

The parse_model module, introduced in dbt-databricks 1.1.0.2, contains functions that can crash when encountering:

1. Macro objects - When the adapter tries to parse Macro objects (which are not actual models), it fails because these objects don't have the expected model attributes
2. Missing configurations - When model configs are incomplete or missing specific attributes, the parsing functions throw exceptions instead of gracefully handling the missing data
3. Unsupported object types - The adapter doesn't correctly handle edge cases where objects passed to it aren't the expected model types

Changing these to a safer implementation solves the issue, as can be seen in this draft PR.

We will check if this is a bug in dbt-databricks or our usage of it, and continue accordingly.

In the meantime, we will use dbt-databricks<1.10.2 as a workaround.

[mi-volodin]

Just to bump up the issue - my company have a critical need to go to dbt-databricks>=1.10.5 or higher, and this issue prevents us from doing so.

I see that PR with a draft fix was already proposed... but since it's been 2 month since it was updated last time - is there any chance that it will be fixed anytime soon? Is there a way to help?

[Jeremynadalmirakl]

Just to bump up the issue - my company have a critical need to go to dbt-databricks>=1.10.5 or higher, and this issue prevents us from doing so.

I see that PR with a draft fix was already proposed... but since it's been 2 month since it was updated last time - is there any chance that it will be fixed anytime soon? Is there a way to help?

Hi @mi-volodin I don't know your setup but as a "patch" on my side, I made the process that generates the elementary report run with an appropriated dbt-databricks while my workflows run with a newer version.... :/

Hope that helps...

[mi-volodin]

Yep, using different venvs might help indeed. Just it's then not easy to deliver this, but could be an option. I was just curious is that would be fixed anytime soon.

[asos-dipeshbhundia]

@arbiv Is there a timeline on when this issue likely to be fixed? dbt-databricks is now on 1.12 and we need the later versions for our project. I don't really want to have to remove elementary

[haritamar]

Hey all, we were a bit cautious about the fix mentioned above because it patches internal dbt-databricks functions. That being said - it looks like the issue is solved in newer versions! So we're removing the restriction and it will be in a version we intend to release today.

[github-actions[bot]]

This issue is stale because it has been open for too long with no activity. If you would like the issue to remain open, please remove the stale label or leave a comment.