spark-standalone-cluster-on-docker icon indicating copy to clipboard operation
spark-standalone-cluster-on-docker copied to clipboard

Published 20 hours ago • verified publisher icon cluster-apps-on-docker

[FEATURE] Ability to split Spark Master and Workers Nodes and run on different cluster nodes

Open alexeyegorov opened this issue 4 years ago • 0 comments

Ability to split Spark Master and Workers Nodes and run on different cluster nodes

Description

As this repository is a really nice start for creating a small Spark cluster on Docker, I found it amazing to be able to use this same set up as a start for distributing this on multiple nodes. I was trying to split up the docker compose into Spark Master + Node1 and Node2 as two Docker instances, but was not able to connect Node2 to Master. Do you think it is somehow interesting?

alexeyegorov avatar Jan 11 '21 14:01 alexeyegorov