dbt-athena
dbt-athena copied to clipboard
dbt docs generate fails with "Column 'table_schema' is ambiguous"
When using the out of the box project from dbt init and dbt-athena (git+https://github.com/daha/[email protected] )
dbt run is successful, but dbt docs generate fails. Copy and paste from Windows 10 / powershell:
PS C:\athena-test> dbt run Running with dbt=0.17.0 Found 2 models, 4 tests, 0 snapshots, 0 analyses, 134 macros, 0 operations, 0 seed files, 0 sources
15:42:52 | Concurrency: 8 threads (target='dev') 15:42:52 | 15:42:52 | 1 of 2 START table model dev.my_first_dbt_model...................... [RUN] 15:42:58 | 1 of 2 OK created table model dev.my_first_dbt_model................. [OK in 6.29s] 15:42:58 | 2 of 2 START view model dev.my_second_dbt_model...................... [RUN] 15:43:03 | 2 of 2 OK created view model dev.my_second_dbt_model................. [OK in 4.48s] 15:43:03 | 15:43:03 | Finished running 1 table model, 1 view model in 16.43s.
Completed successfully
Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2 PS C:\athena-test> dbt docs generate Running with dbt=0.17.0 Found 2 models, 4 tests, 0 snapshots, 0 analyses, 134 macros, 0 operations, 0 seed files, 0 sources
15:43:21 | Concurrency: 8 threads (target='dev') 15:43:21 | 15:43:21 | Done. 15:43:21 | Building catalog Encountered an error while generating catalog: Runtime Error Runtime Error SYNTAX_ERROR: line 42:24: Column 'table_schema' is ambiguous dbt encountered 1 failure while writing the catalog 15:43:22 | Catalog written to C:\athena-test\target\catalog.json
I found the problem!
In this file:
AppData\Local\Programs\Python\Python38\Lib\site-packages\dbt\include\athena\macros\catalog.sql
Change this line:
upper("table_schema") = upper('{{ schema }}'){%- if not loop.last %} or {% endif -%}
to:
upper("tables"."table_schema") = upper('{{ schema }}'){%- if not loop.last %} or {% endif -%}
I would submit a PR but there are some legal issues with my employer that will take a bit to work out. It was just easier to show you what I changed.
Thanks!
-Jeff
@BeantownData Thanks for the workaround! After changing the line I got a different error:
Running with dbt=0.16.1
Found 26 models, 131 tests, 0 snapshots, 2 analyses, 258 macros, 0 operations, 0 seed files, 152 sources
09:30:20 | Concurrency: 8 threads (target='dev')
09:30:20 |
09:30:22 | Done.
09:30:23 | Building catalog
Encountered an error while generating catalog: Runtime Error
Runtime Error
HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: Error: name expected at the position 7 of 'struct<$oid:string>' but '$' is found. (Service: null; Status Code: 0; Error Code: null; Request ID: null)
dbt encountered 1 failure while writing the catalog
@smomni Sorry, I have never seen that error before and I have no idea.
I'm getting the same error as well.
Runtime Error HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: Error: type expected at the position 0 of 'INT' but 'INT' is found. (Service: null; Status Code: 0; Error Code: null; Request ID: null; Proxy: null)