Raja Farrukh Ali

## Motivation EnvPool's [documentation ](https://envpool.readthedocs.io/en/latest/env/procgen.html)does not mention supporting the 'exploration' [distribution mode](https://github.com/sail-sg/envpool/blob/aacf06f694ead2eb75331f085f00dad71eec1a08/envpool/procgen/registration.py#L38) for Procgen. Procgen environment options: https://github.com/openai/procgen#environment-options (although I am not sure if the original Procgen supports it too,...

Value of Epsilon Decay Period

3

In the TF version of DQN, the value of `epsilon_decay_period` is set to 1M steps (see [here](https://github.com/google/dopamine/blob/a2753dae222c75ae991758d4110a84bc01c3215f/dopamine/agents/dqn/configs/dqn_nature.gin#L15)), and for Rainbow, the value is set to 250k steps (see [here](https://github.com/google/dopamine/blob/a2753dae222c75ae991758d4110a84bc01c3215f/dopamine/agents/rainbow/configs/rainbow.gin#L18)). However,...

Hi, Why is the dueling and double_q are both set to false [here](https://github.com/ray-project/rl-experiments/blob/b848a10c08da302febd860270e90672addc42ef5/atari-dqn/dueling-ddqn.yaml#L11)? Seems this is a redundant/old file as there is another file named `duel-ddqn.yaml` in the same folder...

Hi, can anyone explain the difference between train_batch_size and sample_batch_size, used [here](https://github.com/ray-project/rl-experiments/blob/b848a10c08da302febd860270e90672addc42ef5/atari-dqn/dist-dqn.yaml#L22)? I looked up this key for its explanation in [Common Parameters](https://docs.ray.io/en/master/rllib-training.html#common-parameters) and DQN [trainers](https://github.com/ray-project/ray/blob/master/rllib/agents/dqn/dqn.py) but couldn't fine one....