[ZEPPELIN-5986] Re-enable Junit 5 integration tests by upgrading maven plugins

[pan3793]

trafficstars

What is this PR for?

I found that we can not trigger the Livy integration tests while tackling the Livy Hadoop3 upgrading, I finally found it was caused by the low version of maven-failsafe-plugin not supporting JUnit5 annotations.

Actually, maven-surefire-plugin(for UT) and maven-failsafe-plugin(for IT) are developed in the same project[1] and they share the same release cycle thus have same versions.

This PR upgrades these two plugins to the latest 3.2.2 to recover the IT written in Junit5.

[1] https://github.com/apache/maven-surefire

What type of PR is it?

Fix test cases can not run.

Todos

• [ ] - Task

What is the Jira issue?

ZEPPELIN-5986

How should this be tested?

Pass CI

Screenshots (if appropriate)

Questions:

• Does the license files need to update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? No

[pan3793]

alright, all integration test cases were triggered now, but the bad news is some cases are failed :(

[pan3793]

@Reamer @zjffdu @jongyoul I suppose the IT has been broken for a long time, should we merge the plugin upgrading with failure first, or merge after fixing all tests(may take a long time as I'm not very familiar with Python and Selenium)?

[smileyboy2019]

When will the new version be released? It's been a long time since a new version was released

[smileyboy2019]

When will the new version be released? It's been a long time since a new version was released

[pan3793]

When will the new version be released? It's been a long time since a new version was released

It was scheduled at the end of 2023 https://www.mail-archive.com/[email protected]/msg37111.html

[pan3793]

I have no idea why integration tests can not see R packages installed via conda, I'm seeking to use apt to manage R packages in CI.

Error in library("knitr"): there is no package called ‘knitr’

[Reamer]

@pan3793 Have you got the Livy tests running? In my case, either the IDE or Maven crashes completely during the tests (I suspect a SIGKILL in their test framework)

The latest status of your change is as follows.

[INFO] Running org.apache.zeppelin.livy.LivyInterpreterIT
 WARN [2023-12-08 15:40:40,533] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,550] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,558] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,559] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,561] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,563] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,565] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set
 WARN [2023-12-08 15:40:40,567] ({main} LivyInterpreterIT.java[checkPreCondition]:79) - livy integration is skipped because LIVY_HOME is not set

@jongyoul Do we want to remove livy from Zeppelin?

[pan3793]

Have you got the Livy tests running?

@Reamer Sorry for the late reply. I can trigger the Livy IT after exporting LIVY_HOME and SPARK_HOME properly, but all tests failed later ...

During my previous exploration, I found the Livy integration testing is quite heavy and a little bit hacky, it launches an embedded HDFS/YARN cluster with a Livy Server to allow Zeppelin to run integration tests, I'm not sure the GitHub Actions have sufficient resources to bootstrap, and it also depends on Scala 2.11 since Livy does not publish 2.12 jars to Maven central.

[pan3793]

Close and in favor https://github.com/apache/zeppelin/pull/4725