spark-ec2
spark-ec2 copied to clipboard
amplab
Scripts used to setup a Spark cluster on EC2
Connect to worker web broken. From the master UI page, linked to worker page: http://xxxx.us-west-1.compute.amazonaws.com:8081 But it is broken.
Here's a beta version of Ubuntu 14.04 addition to spark-ec2. There are some simplifications made for the Ubuntu create/setup sequence where Rstudio, ganglia, and persistent-hdfs have not been ported to...
There is work to enable multiple Scala versions support. But so far only Scala 2.10.3 is supported. Is there interest and/or a roadmap for Scala 2.11 support?
When I upgraded to Spark 2.0, I took the 2.0 branch from this repository and it allowed me to spin up Spark-2.0 clusters on Amazon EC2. However, there does not...
I am trying to connect to the cluster using sparklyr but I need to define an environment variable: SPARK_HOME. I cannot find any clue as to what spark home should...
whilst starting the cluster it kept getting: Permission denied (publickey). lost connection any ideas how to debug?
It would be excellent to have support for specifying a specific hadoop minor version. Spark itself is distributed for 2.4.0 and 2.6.0, but there is no way to currently use...
I am using branch-2.0.0 and I see it is using: Spark 2.0, Scala 2.10.6. java 1.7 (jre) java 1.6 (jdk) aws cli (0.? I forget which) I spent a little...
Hi, I was trying to use the spark-ec2 script from Spark to create a new Spark cluster wit an user different than root (--user=ec2-user). Unfortunately the part of the script...
It seems like the ami EBS type is set to 'standard' which is a previous generation HDD. This type of EBS Volume costs money for I/O and in that case...
