Adversarial Behavioral Cloning

Improves on the SeqGAN idea by adding more reinfocement learning and GAN techniques like:

• a replay buffer
• Consensus Optimization
• count-based exploration bonus
• Proximal Policy Optimization (was not found to help, but can be found in this commit)
• advantage normalization

How to run

If you wish to enable Consensus Optimization (via the --grad-reg option), you'll need to patch PyTorch to allow forcing the use a twice-differentiable RNN.

python3 main.py will run the project with the default options. Output will be written to the run/ directory.

Shameless plug

The em tool makes it really easy to twiddle hyperparameters by tracking changes to code (no need to make everything an option!).

Just run em run -g 0 exp_name with your desired options and you'll find a reproducable snapshot in experiments/<exp_name>!

If you want to resume from a snapshot (perhaps with different options), use em resume -g 0 exp_name ...

You can also fork an experiment and its changes using em fork, but the quick and dirty solution is to run bash scripts/make_links.sh :)