Lei Wang

how to remain conf upon spark cluster stop/start via spark-ec2

1

Some conf change requires cluster restartup to take effect, say 'SPARK_WORKER_OPTS'; but during stop/start spark cluster via spark-ec2, it seems it re-setup cluster and flushes all of conf. So is...

Is there a handy Hadoop cluster running?

3

With Spark cluster launched and started with ec2 script, is there the Hadoop cluster ready to go, or just an easy command to start it? The reason I am asking...

How can launch Spark 2.0.1 and Hadoop 2.7?

2

Right now it seems the latest hadoop version in spark-ec2 is 2.4, but actually in spark download page, it can be up to 2.7, also it is available in aws...