zeppelin [HELP-WANTED][ZEPPELIN-5976] Support Flink 1.18

What is this PR for?

Notice, that the patch does not work, I encountered an issue that I am unable to resolve, if someone is interested in this task, feel free to take over.

java.lang.RuntimeException: error reading Scala signature of org.apache.flink.table.api.ImplicitExpressionConversions: unsafe symbol x$2 (child of value <local api>) in runtime reflection universe
  at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:44)
  at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:622)

To reproduce the issue in local,

$ ./mvnw -pl flink/flink-scala-2.12 -Pflink-118 -Phadoop3 clean install -DskipTests -am
$ ./mvnw -pl flink/flink-scala-2.12 -Pflink-118 -Phadoop3 test

What type of PR is it?

Feature

Todos

What is the Jira issue?

ZEPPELIN-5976

How should this be tested?

Strongly recommended: add automated unit tests for any new or changed behavior
Outline any manual steps to test the PR here.

Screenshots (if appropriate)

Questions:

Does the license files need to update?
Is there breaking changes for older versions?
Does this needs documentation?

Oct 05 '24 07:10 pan3793

An addition thought, the Flink community decided to deprecate all Scala APIs in Flink 1.18 and remove them in Flink 2.0, see more details at FLIP-265. What kind of Flink APIs you are using? Java, Scala, or Python? IMO the support of Flink Scala API in Zeppelin might be removed eventually.

Oct 05 '24 08:10 pan3793

If we need to change them completely, I prefer to use Java. Of course, other languages are good but our interpreter is based on Java so we'd better do it.