dbt-athena icon indicating copy to clipboard operation
dbt-athena copied to clipboard

dbt docs generate fails with "Column 'table_schema' is ambiguous"

Open BeantownData opened this issue 4 years ago • 4 comments

When using the out of the box project from dbt init and dbt-athena (git+https://github.com/daha/[email protected] )

dbt run is successful, but dbt docs generate fails. Copy and paste from Windows 10 / powershell:

PS C:\athena-test> dbt run Running with dbt=0.17.0 Found 2 models, 4 tests, 0 snapshots, 0 analyses, 134 macros, 0 operations, 0 seed files, 0 sources

15:42:52 | Concurrency: 8 threads (target='dev') 15:42:52 | 15:42:52 | 1 of 2 START table model dev.my_first_dbt_model...................... [RUN] 15:42:58 | 1 of 2 OK created table model dev.my_first_dbt_model................. [OK in 6.29s] 15:42:58 | 2 of 2 START view model dev.my_second_dbt_model...................... [RUN] 15:43:03 | 2 of 2 OK created view model dev.my_second_dbt_model................. [OK in 4.48s] 15:43:03 | 15:43:03 | Finished running 1 table model, 1 view model in 16.43s.

Completed successfully

Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2 PS C:\athena-test> dbt docs generate Running with dbt=0.17.0 Found 2 models, 4 tests, 0 snapshots, 0 analyses, 134 macros, 0 operations, 0 seed files, 0 sources

15:43:21 | Concurrency: 8 threads (target='dev') 15:43:21 | 15:43:21 | Done. 15:43:21 | Building catalog Encountered an error while generating catalog: Runtime Error Runtime Error SYNTAX_ERROR: line 42:24: Column 'table_schema' is ambiguous dbt encountered 1 failure while writing the catalog 15:43:22 | Catalog written to C:\athena-test\target\catalog.json

BeantownData avatar Jul 16 '20 19:07 BeantownData

I found the problem!

In this file:

AppData\Local\Programs\Python\Python38\Lib\site-packages\dbt\include\athena\macros\catalog.sql

Change this line:

          upper("table_schema") = upper('{{ schema }}'){%- if not loop.last %} or {% endif -%}

to:

          upper("tables"."table_schema") = upper('{{ schema }}'){%- if not loop.last %} or {% endif -%}

I would submit a PR but there are some legal issues with my employer that will take a bit to work out. It was just easier to show you what I changed.

Thanks!

-Jeff

BeantownData avatar Jul 17 '20 13:07 BeantownData

@BeantownData Thanks for the workaround! After changing the line I got a different error:

Running with dbt=0.16.1
Found 26 models, 131 tests, 0 snapshots, 2 analyses, 258 macros, 0 operations, 0 seed files, 152 sources

09:30:20 | Concurrency: 8 threads (target='dev')
09:30:20 |
09:30:22 | Done.
09:30:23 | Building catalog
Encountered an error while generating catalog: Runtime Error
  Runtime Error
    HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: Error: name expected at the position 7 of 'struct<$oid:string>' but '$' is found. (Service: null; Status Code: 0; Error Code: null; Request ID: null)
dbt encountered 1 failure while writing the catalog

stumelius avatar Aug 13 '20 06:08 stumelius

@smomni Sorry, I have never seen that error before and I have no idea.

BeantownData avatar Aug 14 '20 12:08 BeantownData

I'm getting the same error as well.

Runtime Error HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: Error: type expected at the position 0 of 'INT' but 'INT' is found. (Service: null; Status Code: 0; Error Code: null; Request ID: null; Proxy: null)

prnjl21 avatar Sep 11 '20 19:09 prnjl21