snowflake-connector-python icon indicating copy to clipboard operation
snowflake-connector-python copied to clipboard

Published 20 hours ago • verified publisher icon snowflakedb

SNOW-1366543: Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise

Open rajathpatel23 opened this issue 9 months ago • 5 comments

We are getting this log error while processing data from Snowflake. It seems to be making a lot of noise; I would like to understand - what was the change made in the snowflake ticket? I feel the logger there is more of debug logger than actual log error.

The changes introduced in this PR seem to be start to give us the log error

Could you help me with this? Also, can you provide more info on the support for pandas that have come with this change?

rajathpatel23 avatar May 04 '24 17:05 rajathpatel23

I tried raising a support ticket - however I could not create one, hence having this issue as a placeholder for now.

rajathpatel23 avatar May 06 '24 03:05 rajathpatel23

Hey Rajat, That error log is indeed very noisy. It is left over from a debug session that I had while working on the PR you mentioned and does not have a lot of value outside of that. I've created a PR here to remove it: #1942

That PR was for adding support for structured types to the pandas_fetch_all function. Currently most nested data returned by the connector comes as json encoded strings, but in the future it will support returning structured data instead. This was one part of the overall change to add that support.

sfc-gh-jrose avatar May 06 '24 16:05 sfc-gh-jrose

hi - thank you for submitting this issue with us.

Can you please elaborate what kind of log messages exactly you mean? Do you have a snippet from a log where the unexpected loglines would be visible ? Please make sure to sanitize them from any account etc. details.

We have some more information about the structured data types mentioned in the fetch_pandas_all changelog entry, please take a peek here if you're interested in this Public Preview feature.

sfc-gh-dszmolka avatar May 06 '24 16:05 sfc-gh-dszmolka

ah, late to the party :)

sfc-gh-dszmolka avatar May 06 '24 16:05 sfc-gh-dszmolka

@sfc-gh-jrose Thank you so much for taking a look at this quickly.

rajathpatel23 avatar May 06 '24 16:05 rajathpatel23

@sfc-gh-jrose Thanks for making this fix, do you know when a release with this fix will be published to PyPi?

parinpshah94 avatar May 21 '24 18:05 parinpshah94

The version for this fix should be published later this week.

sfc-gh-jrose avatar May 21 '24 19:05 sfc-gh-jrose

version 3.10.1 released with the fix

sfc-gh-dszmolka avatar May 22 '24 08:05 sfc-gh-dszmolka