zeppelin icon indicating copy to clipboard operation
zeppelin copied to clipboard

[ZEPPELIN-4871] Fix livy interpreter 0.7.1 donwload link

Open soicem opened this issue 2 years ago • 12 comments

What is this PR for?

During a previous PR related to livy interpreter version upgrade, CI is not working because of unvalid download link of livy interpreter 0.7.1. then, I create a new PR for fixing a download link of livy interpreter zip file.

@jongyoul previous comment

What type of PR is it?

bug

What is the Jira issue?

(https://issues.apache.org/jira/browse/ZEPPELIN-4871)

How should this be tested?

  • Strongly recommended: add automated unit tests for any new or changed behavior
  • Outline any manual steps to test the PR here.

AS IT https://dist.apache.org/repos/dist/release/incubator/livy/0.7.1-incubating/livy-0.7.1-incubating-bin.zip TO BE https://dist.apache.org/repos/dist/release/incubator/livy/0.7.1-incubating/apache-livy-0.7.1-incubating-bin.zip

Questions:

  • Does the licenses files need to update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

soicem avatar Aug 07 '22 15:08 soicem

It looks like spark integration is failed, not sure whether it is related to this PR

zjffdu avatar Aug 08 '22 02:08 zjffdu

It looks like spark integration is failed, not sure whether it is related to this PR

Sometimes, it fails so ...

jongyoul avatar Aug 08 '22 04:08 jongyoul

BTW, I've checked some code and the code looks finding the version of Livy so I think we should change/add some code for supporting a newer version of Livy. @soicem Could you please check the Livy interpreter as well to verify if it needs to change or not?

jongyoul avatar Aug 08 '22 04:08 jongyoul

@soicem ping?

jongyoul avatar Aug 25 '22 12:08 jongyoul

@jongyoul I will verify your feedback on this weekend! thank you for the notification :)

soicem avatar Aug 25 '22 12:08 soicem

I'm writing the test code. After that, i will request a code review ! :)

soicem avatar Aug 27 '22 15:08 soicem

Hello @jongyoul. sorry for late.

I want to add a programmatic interpreter of livy 0.7.1 by referring to existing code. So, I tried to run the existing test code(LivyInterpreterIT.java), checkPreCondition funtion passed, but the code related to cluster below is blocked.

@BeforeClass
  public static void setUp() {
    if (!checkPreCondition()) {
      return;
    }
    cluster = Cluster$.MODULE$.get();

ERROR [2022-08-28 23:49:35,116] ({main} Logging.scala[error]:52) - Failed to initialize cluster. org.scalatest.exceptions.TestFailedDueToTimeoutException: The code passed to eventually never returned normally. Attempted 302 times over 30.083785471 seconds. Last failure message: assertion failed: MiniLivyMain hasn't started yet.. at org.scalatest.concurrent.Eventually$class.tryTryAgain$1(Eventually.scala:420) at org.scalatest.concurrent.Eventually$class.eventually(Eventually.scala:438) at org.scalatest.concurrent.Eventually$.eventually(Eventually.scala:478) at org.scalatest.concurrent.Eventually$class.eventually(Eventually.scala:307) at org.scalatest.concurrent.Eventually$.eventually(Eventually.scala:478) at org.apache.livy.test.framework.MiniCluster.start(MiniCluster.scala:327) at org.apache.livy.test.framework.MiniCluster.runLivy(MiniCluster.scala:254) at org.apache.livy.test.framework.MiniCluster.deploy(MiniCluster.scala:235) at org.apache.livy.test.framework.Cluster$.liftedTree1$1(Cluster.scala:107) at org.apache.livy.test.framework.Cluster$.cluster$lzycompute(Cluster.scala:99) at org.apache.livy.test.framework.Cluster$.cluster(Cluster.scala:97) at org.apache.livy.test.framework.Cluster$.get(Cluster.scala:121) at org.apache.zeppelin.livy.LivyInterpreterIT.setUp(LivyInterpreterIT.java:59) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54) Caused by: java.lang.AssertionError: assertion failed: MiniLivyMain hasn't started yet.

Do you know how to configure cluster? Thank you for the support!

soicem avatar Aug 28 '22 14:08 soicem

@zjffdu Could you please answer the question? I think we need to set up some Livy environment to test it.

jongyoul avatar Aug 29 '22 12:08 jongyoul

@soicem Concerning the error, did you download and set env for Livy in your local machine?

jongyoul avatar Sep 05 '22 03:09 jongyoul

@jongyoul I'll try one more time. thank you !

soicem avatar Sep 05 '22 10:09 soicem

@soicem Any update?

zjffdu avatar Sep 24 '22 03:09 zjffdu

Please rebase your PR to current master.

Reamer avatar Oct 12 '22 14:10 Reamer