databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard

Unable to write list/array type data

Open akhileshpachipulusu-inviz-DataScience opened this issue 1 year ago • 2 comments

I am trying to save a pandas dataframe to a table which contains columns of type list of strings (array). I am doing it using df.to_sql() using sqlalchemy engine. It doesn't seem to be working. I am getting the following error (databricks.sql.exc.NotSupportedError) Could not infer parameter type from value: ['11th gen laptop', 'natural silver laptop', '1tb and 256gb laptop', '6n038pa#acj laptop', 'intel core i3 laptop', 'fhd display laptop', 'intel uhd laptop', '15s-dr3506tu laptop', '8gb ram laptop', 'hp laptop', 'windows 11 home laptop'] - <class 'list'>

When I use use_inline_params=True, I am getting the following error (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter

Details: databricks-sql-connector[sqlalchemy]==3.1.1 Python 3.8.10 Databricks Serverless SQL hive_metastore catalog

Hi @akhileshpachipulusu-inviz-DataScience! Can you please provide some more details to help us with debugging:

  • are you using an AWS or Azure instance?
  • are you using Warehouse or Compute cluster? Which DBR version it runs?
  • please provide a minimal code snippets for both cases (native and inline parameters) that reproduces you issue.

Thank you!

kravets-levko avatar Apr 17 '24 17:04 kravets-levko

When I use use_inline_params=True, I am getting the following error (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter

FYI this won't work since SQLAlchemy always uses server bound parameters. If you attempt this with connector version 2.9.3 it may work better for you, since that version of the SQLAlchemy dialect used inline parameters.

susodapop avatar Jun 12 '24 16:06 susodapop