What's the difference and relation between --executor-cores and -devices in CPU mode?

[dawuchen]

I know --executor-cores is the param of spark-submit to specify the cpu cores per executor .But in CLI tutorial also says -devices is the number of devices (CPU or GPU) per executor,default 1. I want to know what's the difference and relation between them. Besides,in here I know the --executor-cores must to be 1,if the -devices　has been set to 1 by default ,why should I still specify the --executor-cores explicitly? Looking forward to your guidance,thanks!

[junshi15]

-devices is number of GPU devices per executor. it has no effects in the CPU mode.

[guyang88]

@dawuchen Why "--executor-cores" must to be 1 in cpu mode ? Can I make a executor use more cpus?

[dawuchen]

@guyang88 You can check about it in Job stuck in reduce phase #189