spark-MDLP-discretization
spark-MDLP-discretization copied to clipboard
AbstractMethodError in Spark 2.3.0
I would like to discretize the Epsilon dataset that it has 2k features and 400k records. To achieve this, I utilized the Spark 2.3.0. Once I execute the code, I deal with below error. It is noteworthy this error had not occurred when I used Spark 2.2.0, but It seems that the Spark 2.2.0 is not suitable for a high dimensional dataset. If is it possible, please fix this problem.
######################################################### Exception in thread "main" java.lang.AbstractMethodError at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99) at org.apache.spark.mllib.feature.MDLPDiscretizer.initializeLogIfNecessary(MDLPDiscretizer.scala:48) at org.apache.spark.internal.Logging$class.log(Logging.scala:46) at org.apache.spark.mllib.feature.MDLPDiscretizer.log(MDLPDiscretizer.scala:48) at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) at org.apache.spark.mllib.feature.MDLPDiscretizer.logInfo(MDLPDiscretizer.scala:48) at org.apache.spark.mllib.feature.MDLPDiscretizer.runAll(MDLPDiscretizer.scala:106) at org.apache.spark.mllib.feature.MDLPDiscretizer$.train(MDLPDiscretizer.scala:335) at org.apache.spark.ml.feature.MDLPDiscretizer.fit(MDLPDiscretizer.scala:149) #########################################################