ASE
ASE copied to clipboard
nv-tlabs
Initial state distribution bug when resetting the environment
Hello, thank you for the amazing work. I noticed that when resetting the environment, the initial state is different each time even if stateinit is set to Default or Start. When looking deep into the code, it seems that the initial state is different because the positions of the rigid bodies are kept the same as before the reset. In the function _reset_env_tensors(self, env_ids) in humanoid.py, only root state and dof states are being reset. Appearently, the current version of IsaacGym does not allow for setting positions for rigid bodies according to the doc.
Hi Jack,
I figured out the same thing. I think this issue can be fixed by adding these lines in the reset.
player.env.task.gym.simulate(player.env.task.sim) player.env.task.gym.fetch_results(player.env.task.sim, True)
This way we are always getting the new rigid_body values.
I think that this causes another issue since gym.simulate will step all the environments. So maybe it solves the problem for environments that are to be reset, but the other ones that are not done will skip the next state that results from the last action. So for undone environments, the transition will be like: instead of state_t, action_t => reset done environments => state_t+1 it becomes: state_t, action_t => reset done environments and step the simulation => state_t+2 This probably messes up with the dynamics.
Yes, unfortunately this causes issues where sometimes the simulation will be advanced by two timesteps instead of one. This will probably hurt the training process by introducing some randomness into the dynamics. I'm not sure what a good solution is in this case with IsaacGym. I'm open to suggestions!
