Hadoop url not working anymore

[aialenti]

I tried to use yarn-ec2 to start a Hadoop cluster on aws, actually the tool was suggested on the xgboost website.

It gives me an error when it tries to download the Hadoop distribution from this website: http://www.motorlogy.com/

[dimileeh]

I've just had the same problem trying to follow the manual from the xgboost website :(

[berch]

I am having the exact same issue.

[berch]

bootstrap.log

2017-01-03 22:05:43,577 INFO assuming master is myself as ip-172-31-16-218.us-west-2.compute.internal 2017-01-03 22:05:43,577 INFO sudo apt-get -y update 2017-01-03 22:05:56,244 INFO sudo apt-get -y install protobuf-compiler emacs git g++ make python-numpy libprotobuf-dev libcurl4-openssl-dev 2017-01-03 22:07:05,752 INFO wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u40-b26/jdk-8u40-linux-x64.tar.gz 2017-01-03 22:07:23,595 INFO tar xf jdk-8u40-linux-x64.tar.gz 2017-01-03 22:07:26,098 INFO rm -f jdk-8u40-linux-x64.tar.gz 2017-01-03 22:07:26,128 INFO wget http://www.motorlogy.com/apache/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz 2017-01-03 22:07:26,317 ERROR Command wget http://www.motorlogy.com/apache/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz returns 8 2017-01-03 22:07:26,317 ERROR 2017-01-03 22:07:26,317 ERROR --2017-01-03 22:07:26-- http://www.motorlogy.com/apache/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz Resolving www.motorlogy.com (www.motorlogy.com)... 205.186.175.195 Connecting to www.motorlogy.com (www.motorlogy.com)|205.186.175.195|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2017-01-03 22:07:26 ERROR 404: Not Found.

2017-01-03 22:07:26,317 INFO tar xf hadoop-2.6.0.tar.gz 2017-01-03 22:07:26,323 ERROR Command tar xf hadoop-2.6.0.tar.gz returns 2 2017-01-03 22:07:26,324 ERROR 2017-01-03 22:07:26,324 ERROR tar: hadoop-2.6.0.tar.gz: Cannot open: No such file or directory tar: Error is not recoverable: exiting now

2017-01-03 22:07:26,324 INFO rm -f hadoop-2.6.0.tar.gz

[tqchen]

hmm, we nay need to update the URL to a new one, anyone interested in push a PR on this?

[berch]

There are actually couple more problems with the script. The get-select doesn't get the number of slave nodes correctly and environment variables on master also aren't set. I think there is a ticket for the second issue already.