Presets usage

[sonelu]

After #221 the way the presets are created has changed.

This means that it is not possible to write anymore:

from all.presets import atari

agents = [
    atari.dqn(),
    atari.ddqn(),
    atari.c51(),
    atari.rainbow(),
    atari.a2c(),
    atari.ppo(),
]

as suggested in the documentation.

You need to use:

agents = [
    atari.dqn,
    atari.ddqn,
    atari.c51,
    atari.rainbow,
    atari.a2c,
    atari.ppo,
]

One another annoyance is that by default the agents will be instantiated with CUDA support and if that is not available you need to explicitly relocate them by calling:

agents = [
    atari.dqn.device('cpu'),
    atari.ddqn.device('cpu'),
    atari.c51.device('cpu'),
    atari.rainbow.device('cpu'),
    atari.a2c.device('cpu'),
    atari.ppo.device('cpu'),
]

I think it would be much better if we would implement the lower case presets using partial from functools like this (example for dqn):

from functools import partial
...
dqn = partial(PresetBuilder, default_name='dqn', 
                             default_hyperparameters=default_hyperparameters, 
                             constructor=DQNAtariPreset))

Now using the preset is similar to the pre-#221 as in you can write:

atari.dqn()

which will create the default dqn but with the added bonus that you can also supply the extra parameters like this

atari.dqn(device='cpu')

or even nicer:

atari.dqn(device='cuda:2', hyperparameters={'lr':1e-5}, name='dqn_reduced_lr')