ManiSkill icon indicating copy to clipboard operation
ManiSkill copied to clipboard

Leak of GPU memory when using set_environment_map_from_files function

Open lyshuga opened this issue 2 years ago • 1 comments

Hi,

We have observed one issue. After every reset it was applied "set_environment_map_from_files" to generate a new skybox, but unfortunately after every 10-20 reset the amount of GPU memory used increased by 100-200 MB. So, after let's say 100 resets we run out of GPU memory. The function "set_environment_map_from_files" is the wrapper for C++ function, I was wondering if it possible to somehow overcome this issue.

Thanks

lyshuga avatar Aug 10 '23 13:08 lyshuga

Looks like the memory consumption is due to the cached resources. You can release the memory by

env.unwrapped._renderer.clear_cached_resources()

xuanlinli17 avatar Aug 10 '23 18:08 xuanlinli17

Closing this issue as it has gone stale. Feel free to reopen if not.

StoneT2000 avatar Apr 30 '24 05:04 StoneT2000

Looks like the memory consumption is due to the cached resources. You can release the memory by

env.unwrapped._renderer.clear_cached_resources()

@StoneT2000 @xuanlinli17 Hi, Sorry to dig up this old issue. I am facing something similar in that, whenever I make a new environment in a single process, I notice that my VRAM goes up, and it never comes down. Reset doesn't increase VRAM, but making a new env does.

Image

As you can see, I run 60 parallel of env A for a total of 240 run, which is roughly 10 mins. Then I switch to another envs, rinse and repeat. Yet the VRAM also goes up like a staircase. I checked the time and I can confirm that the VARM only goes up when a new env is made

I tried this api but apparently it was not in the codebase anymore. Is there something similar that I can do? If you think this is an entirely different issues, I am more than happy to open a new independent issue.

Thanks in advance

IrvingF7 avatar Apr 24 '25 03:04 IrvingF7

Will address this in the simpler-env repo

StoneT2000 avatar May 02 '25 01:05 StoneT2000