apache
[#1805][part-2] Support Spark4 (Scala 2.13)
What changes were proposed in this pull request?
Adds Spark4 client.
Why are the changes needed?
Spark4 is about to be released. This tests that the existing Spark3 client is compatible with Spark4, and introduces a Spark3 client, which is a mere copy of the Spark3 client. Releasing an explicit Spark4 client will make it more obvious for users to pick the right client for their Spark4 environment.
Fix: #1805
Does this PR introduce any user-facing change?
It adds a Spark4 client, shaded client and integration tests.
How was this patch tested?
Existing Spark3 tests.
How can we fix those nasty Kerberos failures with JDK 17?
java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Message stream modified (41) - Message stream modified)]; Host Details : local host is: "action-host/127.0.0.1"; destination host is: "localhost":40741;
https://github.com/apache/incubator-uniffle/runs/26438380881
Test Results
2 703 files + 62 2 703 suites +62 5h 30m 21s :stopwatch: + 3m 42s 948 tests + 5 941 :white_check_mark: - 1 1 :zzz: ±0 0 :x: ±0 6 :fire: +6 12 164 runs +399 12 142 :white_check_mark: +392 16 :zzz: +1 0 :x: ±0 6 :fire: +6
For more details on these errors, see this check.
Results for commit a6ebf573. ± Comparison against base commit ea1070c6.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
What do you think about the plain copy of Spark3 client?
What do you think about the plain copy of Spark3 client?
It's ok for me.
What do you think about the plain copy of Spark3 client?
How can we fix those nasty Kerberos failures with JDK 17?
java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Message stream modified (41) - Message stream modified)]; Host Details : local host is: "action-host/127.0.0.1"; destination host is: "localhost":40741;https://github.com/apache/incubator-uniffle/runs/26438380881
cc @zuston
From my sight, kerbreos is not ready for the JDK17. Maybe we could disble this in the java17
