Andy Grove
Andy Grove
Documentation has improved a lot over the past year, so I think we can close this issue now and open more specific issues for extra documentation that is desired.
Also, Spark returns timestamp in seconds and Comet returns timestamp in microseconds. ``` // input: 2023-12-31 10:00:00.0, expected: 1.7040456E9, actual: 1.7040456E15 ```
I added `try_cast` to the list, with a link to the cast epic
Arrow 16 has now been released: https://arrow.apache.org/release/16.0.0.html
@viirya Is there anything stopping us from depending on the datafusion repo directly now (once https://github.com/apache/datafusion/pull/10702 is merged) rather than your fork?
The release vote for arrow-rs 52 has started, and there is a draft PR in DataFusion to upgrade to this version: https://github.com/apache/datafusion/pull/10765
> I think, if we add the functionality for string of this format to decimal in arrow-rs, then the above error could be resolved and also this issue will be...
I plan on working on this once https://github.com/apache/datafusion-comet/pull/307 is merged. I will write a criterion microbenchmark and compare the current approach with a macro approach, and look into other optimizations.
> Hi @andygrove I would love to work on this. If no one else is working on it. Thank you @vidyasankarv that would be awesome! Let me know if you...
Thanks for getting this started @holdenk. I have now created https://github.com/apache/datafusion-comet/pull/587 to add experimental support for Spark 3.5.1 (there will be follow up PRs to fix some test failures), so...