Problem with the submitting Spark job of Morpheus Examples jar

[MohamedRagabAnas]

I have the following on my cluster of machines: OS: Centos 7 Spark: version 2.4.3 Scala: Version 2.12.8

I have run the (./gradlew allJar -x test) in the root directory of the Morpheus project with the following error: Task 'allJar' not found in root project 'okapi'. Some candidates are: 'docJar', 'jar', 'jmhJar'.

So I had to run it with 'jar' option and got jar file for all modules inside the /build/libs directory.

I want to submit spark job of one of the examples inside the generated 'morpheus-examples/build/libs/morpheus-examples-0.4.3-SNAPSHOT.jar'

I have put the required jars in the Spark class path (spark/jars) directory, such as:

• morpheus-spark-cypher-0.4.3-SNAPSHOT.jar
• okapi-api-0.4.3-SNAPSHOT.jar
• okapi-relational-0.4.3-SNAPSHOT.jar

but when I submitted the job like this

spark/bin/spark-submit --class 'org.opencypher.morpheus.examples.CaseClassExample' --master local morpheus-examples/build/libs/morpheus-examples-0.4.3-SNAPSHOT.jar

I always face this error:

Exception in thread "main" java.lang.NoSuchMethodError: scala.App.$init$(Lscala/App;)V at org.opencypher.morpheus.util.App.(App.scala:34) ......

This problem I think because the Scala Version and Spark Version comparability but I installed every thing like the versions inside the gradle parameters as stated in the beginning.

thanks in advance for your help !

[soerenreichardt]

Hi Mohamed, you need to put the fat-jar of Morpheus-Spark-Cypher into your classpath which can be found here: https://github.com/opencypher/morpheus/releases/download/0.4.2/morpheus-spark-cypher-0.4.2-all.jar