Notebook 03a, SparkContext() will not run without modifying `/etc/hosts` on OSX.

[vestuto]

trafficstars

Students in your tutorial may encounter this. The following code cell generates an error without updating /etc/hosts

sc = SparkContext('local[4]')

The error is as following:

Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
: java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries!

Adding the following to the end of /etc/hosts enabled me to run the cell successfully:

127.0.0.1 <hostname>

Where <hostname> is the output from calling hostname from the shell.

[vestuto]

UPDATE: regardless of content of /etc/hosts cannot get notebook 06 to run the following cell without the same error as shown above:

sc = SparkContext('spark://schedulers:7077')

[mrocklin]

According to @quasiben and @ahmadia this may be closer to fixed by setting your JAVA_HOME and JRE_HOME environment variables.

[vestuto]

Setting the following in my ~/.bashrc allows me to run pyspark from the conda environment I've set up...

export JAVA_HOME=$(/usr/libexec/java_home)
export JRE_HOME=$JAVA_HOME/jre

... but I still get the same error from notebook 06

[ahmadia]

This was a VPN issue for me. Are you running one? On Mon, Jul 11, 2016 at 10:07 PM Jason Vestuto [email protected] wrote:

Setting the following in my ~/.bashrc allows me to run pyspark from the conda environment I've set up...

export JAVA_HOME=$(/usr/libexec/java_home) export JRE_HOME=$JAVA_HOME/jre

... but I still get the same error from notebook 06

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mrocklin/scipy-2016-parallel/issues/2#issuecomment-231925763, or mute the thread https://github.com/notifications/unsubscribe/AAfRJaSOdIOPoHUkU_ogDvaaw5jLNaTyks5qUwTRgaJpZM4JJ-Mq .

[vestuto]

nope. just trying to run locally on my laptop.