spark-indexedrdd icon indicating copy to clipboard operation
spark-indexedrdd copied to clipboard

Published 20 hours ago • verified publisher icon amplab

NoClassDefFoundError: org/apache/spark/Logging

Open Yishun99 opened this issue 8 years ago • 2 comments

After packaging to .jar file, I add it to intelliJ IDEA as library. And I get this error. "java.lang.NoClassDefFoundError: org/apache/spark/Logging", because of the "Logging" class is "private[spark]".

How could I use IndexedRDD in ide?

Yishun99 avatar Nov 26 '16 05:11 Yishun99

java.lang.NoClassDefFoundError: com/ankurdave/part/ArtTree My problem when put something into indexedRdd

linmingze avatar Feb 28 '17 12:02 linmingze

Don't know about @DouYishun problem, but I was having the same issue as @linmingze. It probably is some problem with dependency resolution. It's basically the same issue described in #8 The workaround I used was building the package myself through the sbt assembly command, so it included all the dependencies in the target .jar, which then I pass as argument when invoking spark-shell or spark-submit.

i.e. what I did was simply:

$ git clone https://github.com/amplab/spark-indexedrdd $ cd spark-indexedrdd $ sbt assembly $ $SPARK_HOME/bin/spark-shell --jars target/scala-2.11/spark-indexedrdd-assembly-0.4.0.jar

luismilanooliveira avatar Apr 05 '17 17:04 luismilanooliveira