SparkInternals icon indicating copy to clipboard operation
SparkInternals copied to clipboard

Published 20 hours ago • verified publisher icon JerryLead

第一章配置多个 CoarseGrainedExecutorBackend 进程

Open wojiaohgl opened this issue 8 years ago • 2 comments

SPARK_WORKER_INSTANCES http://spark.apache.org/docs/latest/spark-standalone.html#cluster-launch-scripts 指定这个参数是不是就可以了？

wojiaohgl avatar Apr 18 '16 08:04 wojiaohgl

SPARK_WORKER_INSTANCES 是配置一个node 上可以运行多少个Worker，一般一个Worker可以运行一个CoarseGrainedExecutorBackend进程，我的问题是如何配置一个Worker可以运行多个CoarseGrainedExecutorBackend

JerryLead avatar May 02 '16 12:05 JerryLead

要想实现一个Worker运行多个CoarseGrainedExecutorBackend，需要将Worker的资源数设置为Executor资源数的N倍

JerryLead avatar May 02 '16 12:05 JerryLead