Procgen Integration

[YukunJ]

Procgen Integration

@author: Yukun J

@date: Apr 18, 2022

@brief: After getting a bit familiar with both the EnvPool and Procgen code bases, make the first draft of Procgen environment integration

@progress: link with the third party procgen environment from OpenAI. The basic logic is implemented. Each Envpool environment will contain an instance of Procgen <Game> class and make appropriate(hopefully) calls to functions when necessary. And modify the project’s bazel file to make it compile successfully.

@todo: haven’t written any cc/py tests yet. Surely I believe tests will prevail with quite a few bugs in the current implementation…But I'm not sure how to write deterministic tests since randomness is involved in each game. Will try to debug and add tests in the following weeks (if final exam workload permits)

@pre: To link the Procgen environment during the compilation, we need the QT library. I refer to https://github.com/justbuchanan/bazel_rules_qt for the bazel rules. However, even if it’s dynamically linked, it still requires that the host machine, who is doing the compilation, has QT5 installed. Fortunately, most Linux and Ubuntu systems do come with the default QT5 already installed.

@file:

• modified: WORKSPACE
• modified: envpool/BUILD
• modified: envpool/entry.py
• modified: envpool/workspace0.bzl
• modified: setup.cfg
• New: envpool/procgen/
• New: third_party/procgen/

@notice: When using “make format”, the following 2 lines in procgen.cc will change order. However, due to some mysterious conflicting stuff in namespace, they have to be declared in the following order, otherwise compilation errors.

#include "envpool/core/py_envpool.h"
#include "envpool/procgen/procgen.h" # this one must come later after py_envpool.h