spark-ec2
spark-ec2 copied to clipboard
amplab
Scripts used to setup a Spark cluster on EC2
I started instances successfully with the ./spark-ec2 script but at the end I don't get the information like http://master-hostname:8080 - I assume because I get `No valid Tachyon version found;...
It looks like spark 2.2.0 is officially released. Is it going to be supported in spark-ec2 shortly?
Throwing this up to temporarily evaluate the diffs; make no promise that I have time to resolve any conflicts. The spark-2.2.0 branch served the purpose for our POF demo project,...
I don't believe that all of the requirements to use `spark-ec2` with an existing VPC & subnet are documented. If I create a VPC & subnet, then use those to...
I wrote a little object-oriented API on top of the spark CLI infrastructure. As far as I can tell, it is back-compatible with the CLI functionality, which is still retained...
If I don't specify "--spark-version", the "/root/spark/" directory contains nothing but "/conf/". When I specify "--spark-version=2.0.0", "--spark-version=2.0.2", "--spark-version=2.1.0", or "--spark-version=2.2.0", the error is like > Don't know about Spark version:...
PR to close issue https://github.com/amplab/spark-ec2/issues/89.
The documentation appears to be incorrect in at least the `branch-1.6` & `branch-2.0` branches. At https://github.com/amplab/spark-ec2#launching-a-cluster, the doc says "Go into the `ec2` directory in the release of Apache Spark...