How can I use more cpus in cpu mode?

Open guyang88 opened this issue 8 years ago • 1 comments

@junshi15 @anfeng I run caffeonspark on cpu grid(32coresX10) .In spark_on_yarn mode ,how many the number of "executor-cores" should I set? I set the number >1,then I got error . And in spark_standalone mode ,how many the number of "core_per_worker" should set? In the same spark_work_instances ,more executor-cores, can task have higher efficiency?

May 19 '17 03:05 guyang88

in YARN mode, spark.executor.cores = 1. https://spark.apache.org/docs/latest/configuration.html

May 22 '17 20:05 junshi15