spark-MDLP-discretization icon indicating copy to clipboard operation
spark-MDLP-discretization copied to clipboard

AbstractMethodError in Spark 2.3.0

Open Majid-Soheili opened this issue 5 years ago • 3 comments

I would like to discretize the Epsilon dataset that it has 2k features and 400k records. To achieve this, I utilized the Spark 2.3.0. Once I execute the code, I deal with below error. It is noteworthy this error had not occurred when I used Spark 2.2.0, but It seems that the Spark 2.2.0 is not suitable for a high dimensional dataset. If is it possible, please fix this problem.

######################################################### Exception in thread "main" java.lang.AbstractMethodError at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99) at org.apache.spark.mllib.feature.MDLPDiscretizer.initializeLogIfNecessary(MDLPDiscretizer.scala:48) at org.apache.spark.internal.Logging$class.log(Logging.scala:46) at org.apache.spark.mllib.feature.MDLPDiscretizer.log(MDLPDiscretizer.scala:48) at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) at org.apache.spark.mllib.feature.MDLPDiscretizer.logInfo(MDLPDiscretizer.scala:48) at org.apache.spark.mllib.feature.MDLPDiscretizer.runAll(MDLPDiscretizer.scala:106) at org.apache.spark.mllib.feature.MDLPDiscretizer$.train(MDLPDiscretizer.scala:335) at org.apache.spark.ml.feature.MDLPDiscretizer.fit(MDLPDiscretizer.scala:149) #########################################################

Majid-Soheili avatar Oct 15 '18 20:10 Majid-Soheili