rerun icon indicating copy to clipboard operation
rerun copied to clipboard

GPU memory not freed when closing recording, but keeping viewer open

Open roym899 opened this issue 1 year ago • 1 comments

Describe the bug It seems like GPU memory is not freed when closing the recording for meshes (only noticed / tested for meshes with large number of vertices, not sure how generally this applies).

As a workaround I have to close the viewer occasionally.

To Reproduce Run the following code a few times while monitoring GPU memory usage. Memory keeps growing and closing the recordings does not free it.

import numpy as np
import rerun as rr

rr.init("Large Mesh", spawn=True)
vertices = np.random.rand(3 * 3000000, 3)
rr.log("mesh", rr.Mesh3D(vertex_positions=vertices))

Expected behavior Memory should be freed when closing a recording in the viewer.

Screenshots

https://github.com/rerun-io/rerun/assets/9785832/0ccee3a2-df56-4cf5-bee4-7f01909adee3

memory Note that "Counted GPU" only goes up, never down.

Backtrace

Desktop (please complete the following information):

  • OS: tested on Ubuntu 22.04 and Arch Linux

Rerun version

rerun-cli 0.17.0-alpha.8 [rustc 1.76.0 (07dca489a 2024-02-04), LLVM 17.0.6] x86_64-unknown-linux-gnu 6649/merge a6a666f, built 2024-06-25T19:47:07Z

Also tested with 0.16.1.

roym899 avatar Jul 04 '24 08:07 roym899

Likely at least partially related to

  • https://github.com/rerun-io/rerun/issues/1962

we're also not good at freeing up anything in that staging belt.

However, one would expect to at least the meshes to disappear from the MeshCache, that's apparently not happening!

Wumpf avatar Jul 04 '24 08:07 Wumpf

Why did #7513 not help with this at all?

teh-cmc avatar Sep 26 '24 07:09 teh-cmc

I though #7531 would fix this issue, however turns out that it is already fixed. The ratched behavior of the CpuWriteGpuRead belt is significant though. That said: for large/many meshes this works out fine now on main

https://github.com/user-attachments/assets/c035c5f4-bb23-4391-98a1-28cc0b5818c9

Wumpf avatar Sep 27 '24 08:09 Wumpf