dbt-snowflake icon indicating copy to clipboard operation
dbt-snowflake copied to clipboard

[Bug] Support Vector data types for macro `snowflake__get_columns_in_relation`

Open chamilton61 opened this issue 7 months ago • 0 comments

Is this a new bug in dbt-snowflake?

  • [X] I believe this is a new bug in dbt-snowflake
  • [X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When trying to run an incremental load referencing a table that has a column that is cast as a VECTOR data type, I get the following error: Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

This appears to be due to the from_description Column function here expecting only integers within parenthesis.

Expected Behavior

Expected column data type to be properly parsed and reproduced in the incremental table.

Steps To Reproduce

do dbt run on a model that has an incremental load materialization and has a column that is the VECTOR data type

Relevant log output

15:40:11  Snowflake adapter: Error running SQL: macro get_columns_in_relation
15:40:11  Snowflake adapter: Rolling back transaction.
15:40:11  On <model name>: Close
15:40:11  Runtime Error in model <model name> (<model name>.sql)
  Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

Environment

- OS: Ubuntu 22.04.4 LTS
- Python:  3.11.7
- dbt-core: 1.8.3
- dbt-snowflake: 1.8.3

Additional Context

suspected cause of the issue: https://github.com/dbt-labs/dbt-snowflake/blob/2576a08379e248d393b9a698e58fac0d32c23648/dbt/include/snowflake/macros/adapters.sql https://github.com/dbt-labs/dbt-adapters/blob/main/dbt/adapters/base/column.py#L130

chamilton61 avatar Jun 28 '24 19:06 chamilton61