[Question] Re-initialize env in direct RL workflow

[guactaco12]

Hi there，

I understand that Isaac Lab does not support dynamically delete and load new assets into the scene during training process. Therefore, I want to train with one set of assets then "clean up" the env. After that, load the second set of assets and training from the saved checkpoint.

Is it feasible to do above in a loop in one python process? I tried to do it in a loop but calling env.close() will lead to stuck if I create env the second time.

Thanks in advance.

[Darth-S1d1ous]

Hi, I'm also new to Isaac Lab but I think I have a workaround here

for iter in range(max_iter):
            ...
            env_cfg = ShadowHandGraspWREnvCfg(
                num_envs=num_envs,
                dataset_dir=self.dataset_dir,
                demo_dir=self.demo_dir,
                object_code=self.obj_code,
                demo_indices=demo_indices,
                action_mode="discrete"
            )
            env = ShadowHandGraspEnv(env_cfg, render_mode=None) # a DirectRLEnv
            ...

            # cleanup
            world = World(env.sim)
            env.close()
            if world.is_playing():
                world.stop()
            world.clear()
            world.clear_instance()
            del env
            ...

This idea is inspired by #2524. I'm not entirely sure about the underlying mechanism behind this workaround — if anyone has a cleaner or more correct approach, please feel free to point it out!

[RandomOakForest]

Thank you for posting this. If you want to train on one set of assets, then "clean up" and load a second set of assets during a training workflow, a typical approach is:

• Create an environment instance once.
• Use environment reset functions along with curriculum managers or environment event managers that support applying new randomized states, procedural changes, or asset swaps on reset.
• Train on the first asset set for some iterations.
• Trigger a reset or task change that loads the second asset set (this could involve reinitializing specific scene elements or using multi-asset spawning capabilities).
• Continue training without calling env.close() and re-instantiating the environment.

[guactaco12]

Thank you for posting this. If you want to train on one set of assets, then "clean up" and load a second set of assets during a training workflow, a typical approach is:

• Create an environment instance once.
• Use environment reset functions along with curriculum managers or environment event managers that support applying new randomized states, procedural changes, or asset swaps on reset.
• Train on the first asset set for some iterations.
• Trigger a reset or task change that loads the second asset set (this could involve reinitializing specific scene elements or using multi-asset spawning capabilities).
• Continue training without calling env.close() and re-instantiating the environment.

Thanks for your reply. Does this mean I can only use ManagerBasedRLEnv instead of DirectRLEnv?