databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard

ValueError time data %r does not match format %r due to fixed formatting in convert_to_assigned_datatypes_in_column_table in v4.0.0

Open RichardOberdieck opened this issue 11 months ago • 2 comments

Issue

SQL query failing due to ValueError when parsing datetime. This is due to the function convert_to_assigned_datatypes_in_column_table, introduced v4.0.0, which takes the fixed format "%Y-%m-%d %H:%M:%S.%f" for the datetime.datetime.strptime() parsing.

Expected setup

Enable a way to be configure the parsing format, or scratch that conversion altogether. In the previous versions, the conversion worked fine.

RichardOberdieck avatar Jan 24 '25 13:01 RichardOberdieck

Seeing the same issue, reverting to 3.7.2 as a workaround

geophpherie avatar Feb 20 '25 20:02 geophpherie

@RichardOberdieck Looking into it, will raise the fix for it. In our 4.x connector PyArrow is an optional library, so you can install PyArrow and then everything should work as it did previously. There is no need to go to the 3.x connector version

jprakash-db avatar Mar 04 '25 09:03 jprakash-db

Any update on this? I just saw you released 4.0.2 but this is not fixed. Shall I make a PR?

RichardOberdieck avatar Apr 09 '25 10:04 RichardOberdieck

Yes, you can raise a PR

jprakash-db avatar Apr 09 '25 16:04 jprakash-db

Was this fixed? We had this same issue with v4.0.0 but it resolved when we upgraded to 4.1.3 (without PyArrow).

pkpenton avatar Sep 24 '25 17:09 pkpenton

Hm, the string format is not fixed anymore as described above, but fromisoformat (see code). We haven't upgraded yet.

RichardOberdieck avatar Sep 25 '25 08:09 RichardOberdieck