dbt-utils icon indicating copy to clipboard operation
dbt-utils copied to clipboard

`not_null_proportion` doesn't work in Athena / Presto due to numeric type

Open SOVALINUX opened this issue 2 years ago • 6 comments

Describe the bug

Athena fails query execution with the error SYNTAX_ERROR: line 15:56: Unknown type: numeric There is indeed no such type in Athena / Presto Suggestion is to use double instead But the issue is actually wider - not_null_proportion hardcoded numeric type instead of using safe_cast with datatypes

Steps to reproduce

Use generic test not_null_proportion with Athena adapter

Expected results

Test works well

Actual results

SYNTAX_ERROR: line 15:56: Unknown type: numeric

System information

The contents of your packages.yml file: Forked dbt-utils from v0.8.4

Which database are you using dbt with?

  • [ ] postgres
  • [ ] redshift
  • [ ] bigquery
  • [ ] snowflake
  • [x] other (specify: Athena)

The output of dbt --version: 1.0.4

Additional context

Are you interested in contributing the fix?

Pull Request with all these fixes will come soon

SOVALINUX avatar Apr 15 '22 13:04 SOVALINUX

Same happening with me, with dbt-utils 0.9.2 and dbt 1.3.0

villasv avatar Nov 10 '22 18:11 villasv

Thanks @SOVALINUX - we don't actively support other warehouse platforms in this repo, but by using the cross-database macros defined in dbt core as of 1.3: https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#type_numeric, you can get the warehouse-specific behaviour you need.

In other words, I'll welcome a PR that changes to using dbt.type_numeric() but not a PR that adds a Presto-specific implementation of that test.

joellabes avatar Nov 25 '22 00:11 joellabes

@joellabes opened https://github.com/dbt-labs/dbt-utils/pull/800 to address this

henriblancke avatar May 23 '23 00:05 henriblancke

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] avatar Nov 19 '23 01:11 github-actions[bot]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

github-actions[bot] avatar Nov 27 '23 01:11 github-actions[bot]

I'm also interested in this fix (working on Trino).

brunokim avatar Mar 08 '24 13:03 brunokim