datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Cast string to timestamp crash while we input time before 1970 with floating number second

Open waitingkuo opened this issue 3 years ago • 2 comments
trafficstars

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

DataFusion CLI v10.0.0
❯ select cast('1969-01-01T00:00:00.1' as timestamp);
thread 'main' panicked at 'invalid or out-of-range datetime', /Users/willy/.cargo/registry/src/github.com-1ecc6299db9ec823/chrono-0.4.19/src/naive/datetime.rs:117:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

while this works

DataFusion CLI v10.0.0
❯ select cast('1969-01-01T00:00:00' as timestamp);
+------------------------------------------------------------------+
| CAST(Utf8("1969-01-01T00:00:00") AS Timestamp(Nanosecond, None)) |
+------------------------------------------------------------------+
| 1968-12-31 16:00:00                                              |
+------------------------------------------------------------------+

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

waitingkuo avatar Aug 08 '22 18:08 waitingkuo

Spent some time tracking this down, root of the problem was on the arrow side doing the conversions. This was not working properly for negative values.

Luckly it looks like it was fixed 4 days ago and in the 20.0.0 release;

https://github.com/apache/arrow-rs/issues/2325

turbo1912 avatar Aug 10 '22 02:08 turbo1912

@turbo1912 thank you! i'll wait for 20.0.0 and write a test case to close this

waitingkuo avatar Aug 10 '22 06:08 waitingkuo