tomesd icon indicating copy to clipboard operation
tomesd copied to clipboard

Published 20 hours ago • verified publisher icon dbolya

Tomesd is not reducing the total memory consumption.

Open ghost opened this issue 2 years ago • 2 comments

When i use Diffusors without tomesd the total memory usage in nvidia-smi is 3966MiB. But when i use it with tomesd ( using the example in the readme ) , the usage on nvidia-smi is still the same.

Any idea whats happening?

ghost avatar Aug 16 '23 16:08 ghost

Yes I experience this too

reynoldscem avatar Oct 31 '23 16:10 reynoldscem

I believe someone looked into this before and found that diffusers was already using memory efficient attention, which saves most of the memory that ToMe would have saved anyway. The numbers in the paper are for the original implementation of stable diffusion.

dbolya avatar Oct 31 '23 16:10 dbolya