`pip install -e .[dev]` installs tons of versions of `awscli`

[dfilan]

Bug description

When installing dependencies in master, seemingly every version of awscli is being installed. See here for illustration.

Steps to reproduce

Run the following:

git clone [email protected]:HumanCompatibleAI/imitation.git
cd imitation/
virtualenv venv
. venv/bin/activate
pip install -e .[dev]

Environment

• Operating system and version: Ubuntu 22.04.1 LTS
• Python version: 3.8.13 and 3.10.5 (tried both)
• Output of pip freeze --all: pip==22.2.2 setuptools==65.0.2 wheel==0.37.1

[dfilan]

@ernestum @AdamGleave I think this involves your recent PR?

[AdamGleave]

This normally happens if pip can't find compatible versions of packages, so tries to install all of them in the hope that one of them works (hint: it usually doesn't).

I wonder if we even need awscli any longer? We were using it to download some pre-trained models (in experiments/imit_benchmark.sh, experiments/train_experts.sh and experiments/download_models.sh).

But the Hugging Face PR replaces the expert part -- so we can actually go ahead and delete experiments/train_experts.sh at least. It doesn't replace the reward models, but I've not been using any of the pre-trained reward models, and at this point they must be quite out of date.

So, I'm in favour of just removing awscli from setup.py and deleting the scripts that use them. (Should update README.md accordingly too.)

@ernestum can you take this on? (Unless Daniel feels like taking it if it's blocking him.)

[dfilan]

Not blocking me, realized that I can just install the Hugging Face package.

[dfilan]

In experiments/imit_benchmark.sh it's just being suggested as a way to upload models, right?

[AdamGleave]

In experiments/imit_benchmark.sh it's just being suggested as a way to upload models, right?

Seems right -- we probably want to delete that as well though if we're not supporting it in download_models.sh