[TASK][EASY] Support JDK 21

[pan3793]

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Search before asking

• [X] I have searched in the issues and found no similar issues.

Skill requirements

• familiar with GitHub Action
• basic knowledge of Apache Maven, including profiles

What would you like to be improved?

Java 21 and the JDK 21 release under a general availability (GA) status occurred Sep 19, 2023, as the newest major update to the Java programming language.

More details on Java 21 (JDK 21) GA release via the JDK-dev mailing list. More details on all of the major improvements with Java 21 can be found via OpenJDK.org.

Kyuubi has supported Java 17 since 1.8, it would be nice if we could support Java 21.

How should we improve?

Step 0: wait for https://github.com/apache/kyuubi/issues/6293 Step 1: test Kyuubi on JDK 21 and fix errors encountered Step 2: add CI in GitHub Actions Step 3(out of scope): leverage new features introduced in JDK 21, e.g. Virtual Thread

Spark 3.5 almost supports Java 21 (except for Arrow, see SPARK-45781), to pass Kyuubi's CI, we need to either

• disable arrow-related tests; or
• replace arrow jars under Spark 3.5 runtime in CI workflow, just as we did for Hive https://github.com/apache/kyuubi/pull/5280

Are you willing to submit PR?

• [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
• [ ] No. I cannot submit a PR at this time.

[Tiakon]

can i work on this issue? please assign me

[pan3793]

@Tiakon thanks assigned

[pan3793]

@Tiakon some additional context, upgrading arrow to 13+ is required to pass all tests with JDK 21, or we can disable all Arrow tests on JDK 21

[Tiakon]

@pan3793 Thanks for reminding me, I will try to modify it and try to pass more tests

[dupen01]

I have recently been tracking this issue and testing the behavior of JDK 21 locally on a branch. The biggest conflict arises from the version of spotless-maven-plugin. The current version of potless-maven-plugin used in the master branch of the Kyuubi project is 2.30.0. To support JDK 21, it is necessary to upgrade to a newer version, as per this issue: https://github.com/diffplug/spotless/issues/1819.

Correspondingly, the version of googlejavaformat also needs to be upgraded (currently, the Kyuubi project uses googlejavaformat v1.7, whereas the latest release of the googlejavaformat project is 1.22.0). This will cause some existing code formatting checks to fail. Furthermore, if upgrading spotless-maven-plugin to a newer version, such as 2.43.0, in order to support JDK 21, it will result in the loss of support for JDK 1.8 (refer to: https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md).

[dupen01]

Perhaps we could maintain the current version of the spotless-maven-plugin by skipping the spotless check process in a JDK 21 environment, just as the Apache Flink community has done.

[pan3793]

@dupen01 for cases you listed, seems could be addressed by overriding properties in java-21 profile

[dupen01]

I will submit a PR for this later.