spark-ec2
spark-ec2 copied to clipboard
tachyon not supported?
I started instances successfully with the ./spark-ec2 script but at the end I don't get the information like http://master-hostname:8080 - I assume because I get No valid Tachyon version found; Tachyon won't be set up
I'm running this on windows, is this not supported?
Is this tachyon necessary, I see on AWS that the instances were started successfully. Is there another way I can get into the web ui?
For some reason the tachyon archive is missing at the url that the script uses.
I guess tachyon is not really necessary. I edited the "./spark_ec2.py" file, and applied these changes:
Old code: modules = ['spark', 'ephemeral-hdfs', 'persistent-hdfs', 'mapreduce', 'spark-standalone', 'tachyon', 'rstudio']
New code: modules = ['spark', 'ephemeral-hdfs', 'persistent-hdfs', 'mapreduce', 'spark-standalone']
After this I encountered no more errors, and I could connect to http://master-hostname:8080.
However, this spark-ec2 project seems to be pretty abandoned. You may be better off trying flintrock.