ImageEnv compatibility with benchmark classes

[MishaLaskin]

Currently, the benchmark tasks on support state observations / goals. However, a lot of RL researchers would benefit from the ability to get image observations / goals. There's an ImageEnv available but it's incompatible with the benchmark classes ML1, etc. It works with single-task environments, but currently there's no goal-setting enabled there.

Would be useful either (a) to have image support for ML1, etc benchmarks or (b) enable goal setting for single task environments.

Is this currently possible? Maybe I missed something in the codebase.

[zhanpenghe]

Hi, thanks for asking! It is possible to use image observations with ML1. Please see https://github.com/rlworkgroup/metaworld/blob/ad807f9d9b6dc577a95ec51795d2b2ed318914fe/tests/metaworld/envs/test_multitask_env.py#L121 as an example. We suggest only use our environment wrapper for meta-learning and multi-task learning. We are recently pushing for the camera ready of this work and will update our guidance after that.

[MishaLaskin]

Thanks @zhanpenghe, indeed getting image observations is straightforward, but the issue is getting goal images. We want to train an agent that is given a goal image (e.g. closed door, stacked block, pushed puck) and achieves the goal using images as observations.

So the question is - how do we render goal images? It seems that the goals are set as the object coordinates, and you want to move the object to those coordinates to set the goal image.

[ryanjulian]

We don't currently have any specific support for retrieving a goal image. This would likely require manipulating the MuJoCo state (into the goal configuration), then rendering the image, then reseting to initial state.

We'd welcome a pull request with this feature, though!

[ryanjulian]

@MishaLaskin we are unlikely to add this ourselves to the current incarnation of the benchmark, because a proper image-conditioned meta-RL or MTRL benchmark would be best if all tasks share a common initial workspace appearance.

Using the disjoint simulations we have today, in most cases the task is trivially-inferable from the initial workspace image.

I will leave this open in case you or someone else would like to contribute an enhancement to support this.

[reginald-mclean]

As Meta-World ownership has been transferred from the RLWorkGroup to Farama Foundation, I am closing this issue. If there are any questions or requests for features please join our Discord