datafusion
datafusion copied to clipboard
Cast string to timestamp crash while we input time before 1970 with floating number second
trafficstars
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior:
DataFusion CLI v10.0.0
❯ select cast('1969-01-01T00:00:00.1' as timestamp);
thread 'main' panicked at 'invalid or out-of-range datetime', /Users/willy/.cargo/registry/src/github.com-1ecc6299db9ec823/chrono-0.4.19/src/naive/datetime.rs:117:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
while this works
DataFusion CLI v10.0.0
❯ select cast('1969-01-01T00:00:00' as timestamp);
+------------------------------------------------------------------+
| CAST(Utf8("1969-01-01T00:00:00") AS Timestamp(Nanosecond, None)) |
+------------------------------------------------------------------+
| 1968-12-31 16:00:00 |
+------------------------------------------------------------------+
Expected behavior A clear and concise description of what you expected to happen.
Additional context Add any other context about the problem here.
Spent some time tracking this down, root of the problem was on the arrow side doing the conversions. This was not working properly for negative values.
Luckly it looks like it was fixed 4 days ago and in the 20.0.0 release;
https://github.com/apache/arrow-rs/issues/2325
@turbo1912 thank you! i'll wait for 20.0.0 and write a test case to close this