SNOW-228474: Support converting VARIANT to python object

[roi-meir]

What is the current behavior?

When querying a variant (or array and object) column the results are returned as dumped json string. In the past before the Arrow format support we used the converter_class parameter to pass our own subclass of SnowflakeConverter that calls json.loads on the returned string. But when using the Arrow format this parameter has no affect

What is the desired behavior?

To be able to control if the results should be a string or a python object (list or dict)

How would this improve snowflake-connector-python?

Either have a way to pass a costume converter class or have another parameter that call json.loads

I will be happy to open a pull request that adds the parameter convert_arrow_variant_to_object that if enabled, call json.loads on variant results.

References, Other Background

[shahargl]

idk if it's relevant for you because it's more of a sqlalchemy thing, but we (@anecdotes.ai) managed to work with DELETE/UPDATE/INSERT/SELECT to VARIANT type columns using sqlalchemy's events mechanism (specifically before_cursor_execute).

[jonashaag]

Any news on this? As a workaround we currently have a hardcoded list of columns to json.parse in the query "postprocessing".

[salexln]

any update on this issue? we are facing the same problems and currently cannot use Arrow because of this

[sfc-gh-mkeller]

Server-side binding for VARIANT and ARRAY is actively in the works. We didn't want to add support for retrieving them until binding them wasn't supported.

[srausser]

any updates/eta's on when we can expect converting VARIANT/ARRAY to python object? can't use snowflake as my feast offline store if the type conversion doesn't happen

[YoniMelki]

any update on this issue? We are facing the same problems and currently cannot use Arrow as well

[sfc-gh-achandrasekaran]

We will look into prioritizing this work in Q2.

[KantorSerhiy]

Hi guys! any update on this issue?

[vanducng]

hello team, any updates?

[sfc-gh-dszmolka]

hi all - to my best knowledge server-side thankfully supports VARIANT and ARRAY now, so next step would be the client side ;) at this moment I don't have any timeline to share for this planned enhancement, but will keep this thread posted with every new information . Thank you so much all for bearing with us !