[Feature Request] Create Preset Atari Environment Configurations

[vwxyzjn]

Motivation

Preprocessing Atari environments is error-prone. E.g., #195.

Solution

Could we simply this by providing an API like envpool.make_atari(style="openai/baseline's") or envpool.make_atari(style="Machado et al 2017")

Typical settings:

# openai/baselines'
envs = envpool.make(
    "Breakout-v5",
    env_type="gym",
    num_envs=8,
    episodic_life=True,
    reward_clip=True,
    seed=1,
    # https://github.com/openai/gym/blob/a7b6462136ebaa610c8941e4da8a9c92155b04d1/gym/envs/__init__.py#L799
    # max_frames = 100000 * 4 = 400000
    max_episode_steps=100000,
    img_height=84,
    img_width=84,
    stack_num=4,
    gray_scale=True,
    frame_skip=4,
    noop_max=30,
    episodic_life=True,
    reward_clip=True,
    repeat_action_probability=0.0,
)

# Machado et al
envs = envpool.make(
    "Breakout-v5",
    env_type="gym",
    num_envs=8,
    episodic_life=False,
    reward_clip=True,
    seed=1,
    # https://arxiv.org/pdf/1710.02298.pdf, Table 3, Max frames per episode
    max_episode_steps=108000 / 4,
    img_height=84,
    img_width=84,
    stack_num=4,
    gray_scale=True,
    frame_skip=4,
    noop_max=0,
    episodic_life=False,
    reward_clip=True,
    repeat_action_probability=0.25,
)

Checklist

• [x] I have checked that there is no similar issue in the repo (required)