[WIP][SPARK-51180][BUILD] Upgrade Arrow to 19.0.0

[aimtsou]

What changes were proposed in this pull request?

This PR aims to upgrade Arrow to 19.0.0 for Apache Spark 4.1.0.

Why are the changes needed?

Bring the latest changes coming from https://arrow.apache.org/release/19.0.0.html

Does this PR introduce any user-facing change?

No

How was this patch tested?

CI

Was this patch authored or co-authored using generative AI tooling?

No

Extra

Keeping for the moment 19.0.0 until 19.0.1 is released.

[aimtsou]

For some reason, the workflow gets forbidden (403) when uploading the image.

[aimtsou]

@dongjoon-hyun: I will look the settings for the fork.

I will update the PR and the ticket on JIRA, and probably hold off until Arrow 19.0.1 since 19.0.0 is not on central maven, and has a small issue which will be corrected on 19.0.1.

[zhengruifeng]

cc @LuciferYang who is also working on arrow upgrade

[LuciferYang]

Starting from 18.2.0, the Java version of Arrow has been moved to a separate repository and is now released independently

• https://github.com/apache/arrow/issues/45364
• https://github.com/apache/arrow-java
• https://github.com/apache/arrow-java/releases/tag/v18.2.0

19.0.0 of Arrow Java has not yet been released now ...

[dongjoon-hyun]

Just a question. Is there any update because we have 19.0.1 already, @aimtsou and @LuciferYang ?

https://arrow.apache.org/release/19.0.1.html

[LuciferYang]

Just a question. Is there any update because we have 19.0.1 already, @aimtsou and @LuciferYang ?

https://arrow.apache.org/release/19.0.1.html

No, the latest arrow-java is still at version 18.2.0

https://github.com/apache/arrow-java/releases

[LuciferYang]

For the record, Apache Arrow 20.0.0 is released with having a new Arrow-Java release. There is no Arrow-Java 19.0.x or 20.0.x.

• https://arrow.apache.org/release/20.0.0.html

test arrow-java 18.3.0 at https://github.com/apache/spark/pull/50892

[github-actions[bot]]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!