ReinforcementLearningEnvironments.jl

⚠️ This package is moved into ReinforcementLearning.jl (2021-05-06)

This package serves as a one-stop place for different kinds of reinforcement learning environments.

Install:

pkg> add ReinforcementLearningEnvironments

API

All the environments here are supposed to have implemented the AbstractEnv related interfaces in ReinforcementLearningBase.jl.

Supported Environments

By default, only some basic environments are installed and exported to make sure that they work on different kinds of platforms. If you want to use some other environments, you'll need to add those dependencies correspondingly.

Built-in Environments

1. MultiArmBanditsEnv
2. RandomWalk1D
3. TigerProblemEnv
4. MontyHallEnv
5. RockPaperScissorsEnv
6. TicTacToeEnv
7. TinyHanabiEnv
8. PigEnv
9. KuhnPokerEnv
10. AcrobotEnv
11. CartPoleEnv
12. MountainCarEnv
13. PendulumEnv

Note: Many traits are borrowed from OpenSpiel.

3-rd Party Environments

Usage

julia> using ReinforcementLearningEnvironments

julia> using ReinforcementLearningBase

julia> env = CartPoleEnv()
# CartPoleEnv

## Traits

| Trait Type        |                  Value |
|:----------------- | ----------------------:|
| NumAgentStyle     |          SingleAgent() |
| DynamicStyle      |           Sequential() |
| InformationStyle  | ImperfectInformation() |
| ChanceStyle       |           Stochastic() |
| RewardStyle       |           StepReward() |
| UtilityStyle      |           GeneralSum() |
| ActionStyle       |     MinimalActionSet() |
| StateStyle        |     Observation{Any}() |
| DefaultStateStyle |     Observation{Any}() |

## Is Environment Terminated?

No

## State Space

`Space{Array{IntervalSets.Interval{:closed,:closed,Float64},1}}(IntervalSets.Interval{:closed,:closed,Float64}[-4.8..4.8, -1.0e38..1.0e38, -0.41887902047863906..0.41887902047863906, -1.0e38..1.0e38])`

## Action Space

`Base.OneTo(2)`

## Current State

[-0.032343893118127506, -0.04221525994544837, 0.024350079684957393, 0.04059943022508135]

julia> state(env)
4-element Array{Float64,1}:
  0.02688439956517477
 -0.0003235577964125977
  0.019563124862911535
 -0.01897808522860225

julia> action_space(env)
Base.OneTo(2)

julia> while !is_terminated(env)
           env(rand(legal_action_space(A)))
       end

julia> using ArcadeLearningEnvironment  # to use 3rd-party environments, you need to manually install and use the dependency first

julia> env = AtariEnv("pong");

Environment Wrappers

Many handy environment wrappers are provided to mimic the OOP style manipulation.

• ActionTransformedEnv
• DefaultStateStyleEnv
• MaxTimeoutEnv
• RewardOverriddenEnv
• StateCachedEnv
• StateOverriddenEnv

Application

Checkout experiments in ReinforcementLearningZoo.jl for how to apply modern reinforcement learning algorithms to these environments. You may also want to read this pluto notebook for how to write a customized environment.