Can not run examples under Spark 2.4

[GangLiCN]

Spark-Bench version (version number, tag, or git commit hash)

2.3.0

Livy Server version

0.5

Details of your cluster setup (Spark version, Standalone/Yarn/Local/Etc)

Spark version: 2.4.0 (installed from "spark-2.4.0-bin-hadoop2.7.tgz") type: Single node, no hadoop, just installed Livy server

Scala version on your cluster

2.11.12

Your exact configuration file (with system details anonymized for security)

The content of "local-livy-example.conf":

spark-bench = { spark-submit-config = [{ livy = { url = "<my_spark_server_ip>:8998" poll-seconds = 1 } spark-bench-jar = "/root/packages/apache/spark/spark-bench_2.3.0_0.4.0-RELEASE/lib/spark-bench-2.3.0_0.4.0-RELEASE.jar" workload-suites = [ { descr = "One run of SparkPi and that's it!" benchmark-output = "console" workloads = [ { name = "sparkpi" slices = 10 } ] } ] }] }

Relevant stacktrace

Exception in thread "main" com.ibm.sparktc.sparkbench.utils.SparkBenchException: REST call returned non-sucess code: 400 at com.ibm.sparktc.sparkbench.sparklaunch.submission.livy.LivySubmit$$anonfun$2.apply(LivySubmit.scala:37) at com.ibm.sparktc.sparkbench.sparklaunch.submission.livy.LivySubmit$$anonfun$2.apply(LivySubmit.scala:37) at com.ibm.sparktc.sparkbench.sparklaunch.submission.livy.LivySubmit$.sendPostBatchRequest(LivySubmit.scala:61) at com.ibm.sparktc.sparkbench.sparklaunch.submission.livy.LivySubmit.launch(LivySubmit.scala:102) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.com$ibm$sparktc$sparkbench$sparklaunch$SparkLaunch$$launch$1(SparkLaunch.scala:57) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$$anonfun$launchJobs$2.apply(SparkLaunch.scala:65) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$$anonfun$launchJobs$2.apply(SparkLaunch.scala:65) at scala.collection.immutable.List.foreach(List.scala:381) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.launchJobs(SparkLaunch.scala:65) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch$.main(SparkLaunch.scala:38) at com.ibm.sparktc.sparkbench.sparklaunch.SparkLaunch.main(SparkLaunch.scala) Caused by: java.lang.Exception at com.ibm.sparktc.sparkbench.utils.SparkBenchException$.apply$default$2(SparkBenchException.scala:22) ... 11 more

Description of your problem and any other relevant info

I can't run example(e.g. local-livy-example.conf) using "spark-bench", detailed error stack is as above. Not sure if it's a compatibility issue between livy server and speark-bench.