dbt-utils
dbt-utils copied to clipboard
`not_null_proportion` doesn't work in Athena / Presto due to numeric type
Describe the bug
Athena fails query execution with the error SYNTAX_ERROR: line 15:56: Unknown type: numeric
There is indeed no such type in Athena / Presto
Suggestion is to use double instead
But the issue is actually wider - not_null_proportion
hardcoded numeric type instead of using safe_cast
with datatypes
Steps to reproduce
Use generic test not_null_proportion
with Athena adapter
Expected results
Test works well
Actual results
SYNTAX_ERROR: line 15:56: Unknown type: numeric
System information
The contents of your packages.yml
file:
Forked dbt-utils from v0.8.4
Which database are you using dbt with?
- [ ] postgres
- [ ] redshift
- [ ] bigquery
- [ ] snowflake
- [x] other (specify: Athena)
The output of dbt --version
:
1.0.4
Additional context
Are you interested in contributing the fix?
Pull Request with all these fixes will come soon
Same happening with me, with dbt-utils 0.9.2 and dbt 1.3.0
Thanks @SOVALINUX - we don't actively support other warehouse platforms in this repo, but by using the cross-database macros defined in dbt core as of 1.3: https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#type_numeric, you can get the warehouse-specific behaviour you need.
In other words, I'll welcome a PR that changes to using dbt.type_numeric() but not a PR that adds a Presto-specific implementation of that test.
@joellabes opened https://github.com/dbt-labs/dbt-utils/pull/800 to address this
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.
I'm also interested in this fix (working on Trino).