Rocky start with dask-ec2 - neither default ami, nor Ubuntu 14.04, 16.04, 17.04 work

[sotte]

Dask-ec2 looks like a gread project! Currently the initial user experience is pretty rocky unfortunately. Neither the default ami, nor amis for Ubuntu 14.04, 16.04, and 17.04 allow me to setup a cluster.

I'll run you through the steps I tried. I used the current version from master 1849efc68545a0ade81b1787da75dcb4df913037 for the experiments.

Default AMI

The default ami ami-d05e75b8 is for us-east-1 and does not work for Europe. Ok, no problem, I'll just use an appropriate ami for Europe (see further down).

Maybe dask-ec2 should suggest a resonable ami based on the region?

Ubuntu 14.04 for Europe

dask-ec2 up \
  --region-name eu-central-1 \
  --keyname some_key_name \
  --tags tag:test \
  --keypair some_key_pair \
  --name some_name \
  --type m4.large \
  --ami ami-ec832a83 \
  --count 4

This runs for a while, creates 4 instances, gives me access to the dask status frontend and the jupyter notebook (via http), but the scheduler does not know about the workers.

Ubuntu 16.04

Doing the same with newer ami: ami-958128fa (Ubuntu 16.04, eu-central-1)

dask-ec2 up \
  --region-name eu-central-1 \
  --keyname some_key_name \
  --tags tag:test \
  --keypair some_key_pair \
  --name some_name \
  --type m4.large \
  --ami ami-958128fa \
  --count 4

This leads to the ssl error #98

...
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2171)

Ubuntu 17.04

Like 16.04, this leads to

...
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2171)

[mrocklin]

Hi @sotte , sorry that you've had a poor experience here. Unfortunately the primary developer of this repository, @danielfrg, is mostly occupied with other things. He might stop by and fix things like this, but we probably can't depend on it. Are you able / interested in helping to resolve some of the issues that you've raised here? If so, that would be greatly appreciated.

[sotte]

Hey @mrocklin, thanks for the quick reply. I'm not complaining about dask-ec2 :) It looks like a great project. I was just trying to summarize the problem I had when checking out the project hoping that it would help to fix some of the issues.

Sadly I don't have the time to help out nor do I have any salt experience.

[jamesbehzadi]

Hi @mrocklin @danielfrg

I'm a big fan of Dask. Works seamlessly on my local machine. But when I try to use dask-ec2 in Australia (ap-southeast-2), I pretty much ran into the same issues as @sotte.

I'd appreciate it, if you could please provide some help. Thanks!

---

Some details:

• Using "ami-d156bab3" (Deep Learning AMI CUDA 9) Bootstrapping salt master fails:

Bootstraping salt master DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 1/3 of function '__install_salt_master' failed DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 2/3 of function '__install_salt_master' failed DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 3/3 of function '__install_salt_master' failed ERROR: 'str' object has no attribute 'decode' Couldn't bootstrap salt-master. Error is above (maybe try again)

• Using "ami-34594757" (Ubuntu Server 14.04 LTS) I received the following CherryPy error:

DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 1/3 of function '__install_salt_rest_api' failed DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 2/3 of function '__install_salt_rest_api' failed DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 3/3 of function '__install_salt_rest_api' failed ERROR: 'str' object has no attribute 'decode' Couldn't install CherryPy. Error is above (maybe try again)

• Using "ami-e2021d81" (Ubuntu Server 16.04 LTS) and "ami-53d4a269" (Ubuntu 14.04.2 LTS) I received the same CherryPy error above.