Security groups leave cluster open to crypto exploits

[thvasilo]

Hello @tqchen

I understand this repo is in maintenance mode now, but I wanted to point out that the security groups that it creates are too loose, exposing the yarn cluster to attacks.

I tried launching a cluster on EMR using these security groups and after a while I noticed jobs being submitted by "dr.who" and after investigation I found that a cryptominer was being run (using a conf file in /var/tmp/w attached below).

I'd recommend tightening the security as the current settings can pose a risk to users.

{
    "algo": "cryptonight",
    "background": true,
    "colors": false,
    "retries": 5,
    "retry-pause": 5,
    "donate-level": 1,
    "syslog": false,
    "log-file": null,
    "print-time": 60,
    "av": 0,
    "safe": false,
    "max-cpu-usage": 95,
    "cpu-priority": 4,
    "threads": null,
    "pools": [
         {
            "url": "stratum+tcp://163.172.205.136:3333",
            "user": "46CQwJTeUdgRF4AJ733tmLJMtzm8BogKo1unESp1UfraP9RpGH6sfKfMaE7V3jxpyVQi6dsfcQgbvYMTaB1dWyDMUkasg3S",
            "pass": "h",
            "keepalive": true,
            "nicehash": false,
            "variant": -1
        }
    ],
    "api": {
        "port": 0,
        "access-token": null,
        "worker-id": null
    }
}

[tqchen]

@thvasilo Thanks for reporting this, can you try to submit a patch to this repo?

[thvasilo]

Hello @tqchen,

I started some work on bringing the code up-to-date with current EC2 standards/software here but I've run into trouble getting the HDFS/YARN distribution to work.

My recommendation to new users is to launch an EMR cluster with HDFS to avoid this complicated setup step and install/build XGBoost locally. I'm creating an automation script for that purpose currently.

As for simply changing the security groups in this repo I'll see if I can copy over the EMR-generated security groups and adjust them to work.