spark-on-kubernetes-helm icon indicating copy to clipboard operation
spark-on-kubernetes-helm copied to clipboard

Published 20 hours ago • verified publisher icon JahstreetOrg

Question: Livy Scala Api

Open jdcohen220 opened this issue 3 years ago • 1 comments

What version of scala/spark do client applications need to use? The image says Spark 3.0.1 but Livy api requires Scala 2.11 which only works with earlier versions of Spark.

I am getting a ClassNotFound error when trying to submit a job: 00:20 WARN: [kryo] Unable to load class org.apache.livy.scalaapi.LivyScalaClient$$anon$1 with kryo's ClassLoader. Retrying with current.. 21/08/23 19:14:12 INFO JobWrapper: Failed to run job b3353a3c-2140-4d4c-94c2-520473152ec2 org.apache.livy.shaded.kryo.kryo.KryoException: Unable to find class: org.apache.livy.scalaapi.LivyScalaClient$$anon$1

jdcohen220 avatar Aug 23 '21 19:08 jdcohen220