spark-sql-perf
spark-sql-perf copied to clipboard
Spark 3.0.0 compile error
Build errors. spark 3.0.0
build.sbt snippet ... crossScalaVersions := Seq("2.11.12", "2.12.10") ... sparkVersion := "3.0.0"
[info] Compiling 66 Scala sources to /opt/spark-sql-perf/target/scala-2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.10. Compiling...
[info] Compilation completed in 18.491 s
[error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Benchmark.scala:338: value table is not a member of Seq[String]
[error] case UnresolvedRelation(t) => t.table
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:59: value table is not a member of Seq[String]
[error] tableIdentifier.table
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:94: missing argument list for method simpleString in class QueryPlan
[error] Unapplied methods are only converted to functions when a function type is expected.
[error] You can make this conversion explicit by writing simpleString _ or simpleString(_) instead of simpleString.
[error] messages += s"Breakdown: ${node.simpleString}"
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:107: missing argument list for method simpleString in class QueryPlan
[error] Unapplied methods are only converted to functions when a function type is expected.
[error] You can make this conversion explicit by writing simpleString _ or simpleString(_) instead of simpleString.
[error] node.simpleString.replaceAll("#\d+", ""),
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:62: not enough arguments for constructor NaiveBayesModel: (uid: String, pi: org.apache.spark.ml.linalg.Vector, theta: org.apache.spark.ml.linalg.Matrix, sigma: org.apache.spark.ml.linalg.Matrix)org.apache.spark.ml.classification.NaiveBayesModel.
[error] Unspecified value parameter sigma.
[error] val model = new NaiveBayesModel("naivebayes-uid", pi, theta)
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:163: not enough arguments for method getCalculator: (impurity: String, stats: Array[Double], rawCount: Long)org.apache.spark.mllib.tree.impurity.ImpurityCalculator.
[error] Unspecified value parameter rawCount.
[error] ImpurityCalculator.getCalculator("variance", Array.fillDouble(0.0))
[error] ^
[error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:165: not enough arguments for method getCalculator: (impurity: String, stats: Array[Double], rawCount: Long)org.apache.spark.mllib.tree.impurity.ImpurityCalculator.
[error] Unspecified value parameter rawCount.
[error] ImpurityCalculator.getCalculator("gini", Array.fillDouble(0.0))
[error] ^
[error] 7 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 178 s, completed Aug 12, 2020, 4:00:12 AM