spark-as-service-using-embedded-server icon indicating copy to clipboard operation
spark-as-service-using-embedded-server copied to clipboard

Published 20 hours ago • verified publisher icon spoddutur

org.apache.spark.SparkException: Could not parse Master URL: 'yarn'

Open quydx opened this issue 4 years ago • 0 comments

When I run app with --master yarn, I got this error

Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'yarn'
        at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2784)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
        at com.spoddutur.spark.SparkFactory$.<init>(SparkFactory.scala:14)
        at com.spoddutur.spark.SparkFactory$.<clinit>(SparkFactory.scala)
        ... 57 more

Was I wrong somewhere ?

quydx avatar Sep 22 '20 07:09 quydx