Strange view from env.reset()

[SeanNobel]

Hi,

When I create an environment simply with env = simpler_env.make("google_robot_pick_coke_can", render_mode="rgb_array"), I get a normal view from above the table like the first image with

obs, _ = env.reset()
obs = get_image_from_maniskill2_obs_dict(env, obs)

but with env.render(), I get the second strange view.

Is that an expected behavior, and how can I render from the same angle as env.reset()?

Thanks.

[xuanlinli17]

This is unexpected as the camera should look at the table even if w/o background overlay; so idk what's happening. You can checkout the sapien viewer.

[SeanNobel]

Thanks for the reply.

By sapien viewer you mean the interactive one by setting render_mode="human", correct? But what about when I want to just simply log evaluation videos during training? Am I wrong that we need render_mode="rgb_array" for that purpose.

Trying out other environments but having the same issue.

As before, the code to reproduce is simply:

env = simpler_env.make("widowx_spoon_on_towel", render_mode="rgb_array")
obs, reset_info = env.reset()
mediapy.show_image(env.render())

If nobody can reproduce this, I think I will redo my installation. Or might do some investigations into the source code... Thanks.