Allow to work OpenAI Gym

[icoxfog417]

Is your feature request related to a problem? Please describe.

As a reinforcement learning researcher/developer, needs to use OpenAI Gym to test the algorithms because it is the common library for reinforcement learning (4000+ citation and 27000+ stars in GitHub). Studio Lab does not have enough libraries to render the OpenAI Gym environments such as BipedalWalker-v2 that depends on BoX2D.

Describe the solution you'd like Pre-install the enough libraries for OpenAI Gym or allows to install libraries by apt. From the Dockerfile for OpenAI Gym test environment, we will need the following libraries.

apt-get -y update && apt-get install -y unzip libglu1-mesa-dev libgl1-mesa-dev libosmesa6-dev xvfb patchelf ffmpeg cmake swig

Describe alternatives you've considered We can install some libraries from conda, but it did not work successfully.

For example, the following is the simple BipedalWalker-v3 test code.

import gym
import numpy as np


env = gym.make("BipedalWalker-v3")

for trial in range(1):
    observation = env.reset()
    done = False
    limit = 10
    step = 0
    while not done or step < limit:
        env.render()
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        print(f"Step {step}: reward = {reward}")
        step += 1

This code causes the following error despite installing freeglut, ffmpeg, swig from conda. And we will need xvfb to render the screen of environment.

Traceback (most recent call last):
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 27, in <module>
    from pyglet.gl import *
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/__init__.py", line 95, in <module>
    from pyglet.gl.gl import *
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/gl.py", line 45, in <module>
    from pyglet.gl.lib import link_GL as _link_function
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib.py", line 149, in <module>
    from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib_glx.py", line 46, in <module>
    glu_lib = pyglet.lib.load_library('GLU')
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/lib.py", line 168, in load_library
    raise ImportError('Library "%s" not found.' % names[0])
ImportError: Library "GLU" not found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/studio-lab-user/rl.py", line 13, in <module>
    env.render()
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/core.py", line 295, in render
    return self.env.render(mode, **kwargs)
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/box2d/bipedal_walker.py", line 491, in render
    from gym.envs.classic_control import rendering
  File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 29, in <module>
    raise ImportError(
ImportError: 
    Error occurred while running `from pyglet.gl import *`
    HINT: make sure you have OpenGL installed. On Ubuntu, you can run 'apt-get install python-opengl'.
    If you're running on a server, you may need a virtual frame buffer; something like this should work:
    'xvfb-run -s "-screen 0 1400x900x24" python <your_script.py>'

The definition of rl environment to run the test code is the following.

name: rl
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - aom=3.3.0=h27087fc_1
  - bzip2=1.0.8=h7f98852_4
  - ca-certificates=2022.5.18.1=ha878542_0
  - cloudpickle=2.1.0=pyhd8ed1ab_0
  - ffmpeg=5.0.1=habc3f16_3
  - freeglut=3.2.2=h9c3ff4c_1
  - freetype=2.10.4=h0708190_1
  - gettext=0.19.8.1=h73d1719_1008
  - gmp=6.2.1=h58526e2_0
  - gnutls=3.7.6=hbf5b4be_4
  - gym-box2d=0.21.0=py310hff52083_2
  - icu=70.1=h27087fc_0
  - lame=3.100=h7f98852_1001
  - ld_impl_linux-64=2.36.1=hea4e1c9_2
  - libblas=3.9.0=15_linux64_openblas
  - libcblas=3.9.0=15_linux64_openblas
  - libdrm=2.4.111=h166bdaf_0
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=12.1.0=h8d9b700_16
  - libgfortran-ng=12.1.0=h69a702a_16
  - libgfortran5=12.1.0=hdcd56e2_16
  - libgomp=12.1.0=h8d9b700_16
  - libiconv=1.16=h516909a_0
  - libidn2=2.3.2=h7f98852_0
  - liblapack=3.9.0=15_linux64_openblas
  - libnsl=2.0.0=h7f98852_0
  - libopenblas=0.3.20=pthreads_h78a6416_0
  - libpciaccess=0.16=h516909a_0
  - libpng=1.6.37=h21135ba_2
  - libstdcxx-ng=12.1.0=ha89aaad_16
  - libtasn1=4.18.0=h166bdaf_1
  - libunistring=0.9.10=h7f98852_0
  - libuuid=2.32.1=h7f98852_1000
  - libva=2.14.0=h7f98852_0
  - libvpx=1.11.0=h9c3ff4c_3
  - libxcb=1.13=h7f98852_1004
  - libxml2=2.9.14=h22db469_0
  - libzlib=1.2.12=h166bdaf_0
  - ncurses=6.3=h27087fc_1
  - nettle=3.8=hc379101_0
  - openh264=2.1.1=h780b84a_0
  - openssl=3.0.3=h166bdaf_0
  - p11-kit=0.23.21=hc5aa10d_4
  - pip=22.1.2=pyhd8ed1ab_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - pyopengl=3.1.6=pyhd8ed1ab_1
  - python=3.10.4=h2660328_0_cpython
  - python_abi=3.10=2_cp310
  - readline=8.1.2=h0f457ee_0
  - sqlite=3.38.5=h4ff8645_0
  - svt-av1=1.1.0=h27087fc_1
  - tk=8.6.12=h27826a3_0
  - tzdata=2022a=h191b570_0
  - wheel=0.37.1=pyhd8ed1ab_0
  - x264=1!161.3030=h7f98852_1
  - x265=3.5=h924138e_3
  - xorg-fixesproto=5.0=h7f98852_1002
  - xorg-inputproto=2.3.2=h7f98852_1002
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libx11=1.7.2=h7f98852_0
  - xorg-libxau=1.0.9=h7f98852_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h7f98852_1
  - xorg-libxfixes=5.0.3=h7f98852_1004
  - xorg-libxi=1.7.10=h7f98852_0
  - xorg-xextproto=7.3.0=h7f98852_1002
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.5=h516909a_1
  - zlib=1.2.12=h166bdaf_0
  - pip:
    - box2d-py==2.3.8
    - future==0.18.2
    - gym==0.21.0
    - numpy==1.22.4
    - pyglet==1.5.26
    - setuptools==62.3.3

Additional context

This issue came from https://github.com/aws/studio-lab-examples/issues/118

[RayYoh]

As a robotics student, I need to use mujoco_py, which also needs to install some additional dependencies via apt install, how should I deal with this problem?

[icoxfog417]

@RayYoh Thank you for the comment. Would you please tell us what apt install libraries does mujoco_py need? From the OpenAI test environment, we estimated that the following libraries are required.

• unzip
• libglu1-mesa-dev
• libgl1-mesa-dev
• libosmesa6-dev
• xvfb
• patchelf
• ffmpeg
• cmake
• swig

[RayYoh]

It could be these, but I'm not sure, because I have different problems every time I install mujoco_py, and its dockfile is here (libgl1-mesa-dev \ libgl1-mesa-glx \ libglew-dev). In addition, I found out yesterday that mujoco_py will not continue to be maintained. DeepMind will use MuJoCo Python Bindings instead. The tutorial is here (it seems to only need ffmpeg).