Could not initialize class org.opencypher. morpheus.impl.MorpheusFunctions

[lakshmanok]

The following code, based on https://github.com/opencypher/morpheus/blob/0.4.2/morpheus-examples/src/main/scala/org/opencypher/morpheus/examples/CypherSQLRoundtripExample.scala :

val result = graph.cypher("""
  |MATCH
  | (t1:Trip)-[:CONTAINS]-(s1:Stop)
  |RETURN t1.id AS trip_id
""".stripMargin)
result.records.asMorpheus.df.toDF("trip_id").createOrReplaceTempView("results")

gives me this error: error: value asMorpheus is not a member of result.Records

[lakshmanok]

Looking at this further, it seems to be a problem with some dependency not being loaded. I'm running the program from the Spark Scala shell as folllows:

spark-shell --packages=org.opencypher:morpheus-spark-cypher:0.4.2

The output indicates that Morpheus is getting installed and I can use classes like MorpheusNodeTable without any problem.

However, doing "result.records" throws this initialization error:

java.lang.NoClassDefFoundError: Could not initialize class org.opencypher.
morpheus.impl.MorpheusFunctions$
  at org.opencypher.morpheus.impl.SparkSQLExprMapper$RichExpression.nullSa
feConversion(SparkSQLExprMapper.scala:66)
  at org.opencypher.morpheus.impl.SparkSQLExprMapper$RichExpression.asSpar
kSQLExpr(SparkSQLExprMapper.scala:97)
  at org.opencypher.morpheus.impl.table.SparkTable$DataFrameTable.$anonfun
$withColumns$2(SparkTable.scala:85)
  at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala
:126)
  at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scal
a:122)
  at scala.collection.immutable.List.foldLeft(List.scala:89)
  at org.opencypher.morpheus.impl.table.SparkTable$DataFrameTable.withColu
mns(SparkTable.scala:84)
  at org.opencypher.morpheus.impl.table.SparkTable$DataFrameTable.withColu
mns(SparkTable.scala:53)
  at org.opencypher.okapi.relational.impl.operators.AddInto._table$lzycomp
ute(RelationalOperator.scala:262)
  at org.opencypher.okapi.relational.impl.operators.AddInto._table(Relatio
nalOperator.scala:260)
...

I am able to do import org.opencypher.morpheus.impl.MorpheusFunctions so the problem is likely with some static initialization that this does, and is not captured in the Maven package ...

[MarcianoAvihay]

Hi ... i suggest not using the maven package, and instead using the FatJar latest release from this repository / building this repo ( i initially started with the fat jar and had no issues , and later switched to building myself - for adding spark 3 and solving some issues / bugs iv'e encountered)

[lakshmanok]

Okay, found the problem. The problem is with line 53 of the MorpheusFunctions initalizer:

https://github.com/opencypher/morpheus/blob/master/morpheus-spark-cypher/src/main/scala/org/opencypher/morpheus/impl/MorpheusFunctions.scala#L53

Start the shell with

spark-shell --packages=org.opencypher:morpheus-spark-cypher:0.4.2 --conf spark.sql.legacy.allowUntypedScalaUDF=True

to workaround this problem.

Another way to set this flag is to do:

spark.conf.set("spark.sql.legacy.allowUntypedScalaUDF", true)