[HUDI-6296] Add Scala 2.13 support for Spark 3.5 integration

[yihua]

Change Logs

As above. Still WIP.

Impact

Adds support for Hudi on Spark 3.5 with Scala 2.13.

Risk level

low

Documentation Update

README.md is updated to include Scala 2.13 build process. HUDI-7695 to track the website docs update.

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[hudi-bot]

CI report:

• edf2bf30a2ddbd48db9452f34b1ac716bd2ebe18 UNKNOWN
• b1598f5861c2b90da91ad33dc360533728ef7163 UNKNOWN
• 9a2450a1bb4454ddc2c86791ce112201f431627a UNKNOWN
• 8105ef96648ad16ec61237a974dbed9e6a2d2c8f Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[ksoullpwk]

Do we have a plan to apply the Scala 2.13 support for older Spark versions?

[yihua]

Do we have a plan to apply the Scala 2.13 support for older Spark versions?

We can add Scala 2.13 support for older Spark versions. Which Spark version do you need support?

[ksoullpwk]

Do we have a plan to apply the Scala 2.13 support for older Spark versions?

We can add Scala 2.13 support for older Spark versions. Which Spark version do you need support?

I'm currently running Hudi with Spark 3.3. It can't be upgraded to higher version now due to another library dependency. It would be better if you can also support Scala 2.13 for Spark 3.3.

[yihua]

Do we have a plan to apply the Scala 2.13 support for older Spark versions?

We can add Scala 2.13 support for older Spark versions. Which Spark version do you need support?

I'm currently running Hudi with Spark 3.3. It can't be upgraded to higher version now due to another library dependency. It would be better if you can also support Scala 2.13 for Spark 3.3.

Got it. It should be straightforward to Support Scala 2.13 for Spark 3.3 integration. Created this ticket to track the work: HUDI-7736.