Please add a button to unload models

[Aamir3d]

When moving between tabs (Bark, AudioGen etc.), each model starts occupying GPU memory and then it starts running out of GPU memory. A good to have would be a button that clears/unloads a model on each page (or combined).

[rsxdalv]

Added for bark. I tested and managed to reclaim some of the GPU memory, but not all of it (for example, I saw 4.5gb -> 1.3 gb drop). But when using CPU offloading, there's almost no reclaimed memory, only about 300mbs (usage goes from 1gb -> 700mb).

[Aamir3d]

Added for bark. I tested and managed to reclaim some of the GPU memory, but not all of it (for example, I saw 4.5gb -> 1.3 gb drop). But when using CPU offloading, there's almost no reclaimed memory, only about 300mbs (usage goes from 1gb -> 700mb).

Thank you! What happens is if you shift to the audiogen section, it starts to use all available GPU memory (12+ VRAM) and there's no way to release memory unless the app is closed and restarted. I'll test the update today for Bark and let you know how it goes.

[rsxdalv]

Yes audiogen is a beast, it's kind of at the limit of "consumer" hardware. I think you can get some of it back if you select the smaller model. But multi band diffusion eats up a lot as well. If it has not been done yet, AudioGen should get CPU offloading.

On Sat, Sep 9, 2023, 11:16 PM Aamir3d @.***> wrote:

Added for bark. I tested and managed to reclaim some of the GPU memory, but not all of it (for example, I saw 4.5gb -> 1.3 gb drop). But when using CPU offloading, there's almost no reclaimed memory, only about 300mbs (usage goes from 1gb -> 700mb).

Thank you! What happens is if you shift to the audiogen section, it starts to use all available GPU memory (12+ VRAM) and there's no way to release memory unless the app is closed and restarted. I'll test the update today for Bark and let you know how it goes.

— Reply to this email directly, view it on GitHub https://github.com/rsxdalv/tts-generation-webui/issues/162#issuecomment-1712592654, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTRXI4R5JS4OEXBQTEWHGDXZTFADANCNFSM6AAAAAA4NMBI4U . You are receiving this because you commented.Message ID: @.***>

[Aamir3d]

I just tested with the latest update. After clicking Unload Models, the memory usage goes down by only 2 GB. Not sure why the remaining memory isn't cleared.

Question: What does the "Clear Models" on the Bark Voice Clone do? Is it similar to Unloading?

Bug : After clicking Unload Models, the button says "Unloaded" and doesn't reset