cleanrl

cleanrl copied to clipboard

Problem Description

A much requested inclusion in the library is to add unit tests. Among other things, "the key benefit of unit tests is to make sure the logic is correct, rather than just looking at black box outputs (does it run? does the loss go down?)"

As such, it is desirable to include unit tests to CleanRL, but I would like to done it in a way that does not change our current file structure too much.

Possible Solution

One possible solution is to leverage runpy like shown in #110. We would basically create a custom gym environment that we specifically define the rewards, states, etc, and then use runpy to run our PPO or DQN scripts. Finally we can extract variables from the runpy runs to do unit tests.