Training RL policies with multi-modality image sensors

[melfm]

Hey guys,

What's the best practice to render multi-modality (rgb, normals etc) + applying visual domain randomization? Since iGibson is faster than NVISII, it makes sense to switch from mujoco-py to iGibson and render multiple modalities using the same environment instance. However just quickly playing with the code, without digging too much into the DR wrappers, I noticed some oddities:

• When rendering with iGibson, the first rendered frame is an empty scene and the robot then suddenly appears (is this a bug? it can be weird when resetting episodes and training the policy)
• I'm not sure if the domain randomization works best with mujoco-py only or with iGibson too ? When I try the DomainRandomizationWrapper I don't see any visual randomization being applied. Furthermore, it slows down the rendering significantly.
• When I enable dynamics randomization, I see some earthquake behavior, the scene starts shaking, is this expected behavior? I think training RL in non-earthquaky scenes is hard-enough already :stuck_out_tongue:

So if these issues are not easy fixes in iGibson, would it make sense to create a RGB randomized env in mujoco-py and use iGibson for other modalities then? I was slightly worried about whether the policy actions executed in separate environments would yield the exact same results ? And the computation speed would be perhaps another issue.

https://user-images.githubusercontent.com/1942293/178534211-02911e10-61c6-41f1-9a4e-69d0f9df3d6b.mp4

[melfm]

So I tried going with the solution of creating two instances, one with mujoco-py and one with iGibson but looks like you cannot create two instances of these two - I get GL context instantiation errors. I didn't have this problem with NVISII, I was able to create two separate environments :disappointed:

Creating offscreen glfw
X Error of failed request:  BadAccess (attempt to access private resource denied)
  Major opcode of failed request:  151 (GLX)
  Minor opcode of failed request:  5 (X_GLXMakeCurrent)
  Serial number of failed request:  163
  Current serial number in output stream:  163