Use "Shared GPU memory"?

[NaughtDZ]

Admittedly, most ordinary users may only have 4-8GB of GPU memory, but there is usually enough shared GPU memory. If using the shared part only reduces the performance, I think most people should accept it. After all, it is better than exceeding the memory limit to cause a crash. Or does the principle of stable diffusion not allow the use of shared GPU memory?

[cch9897]

This may CUDA limit? and RAM to GPU need through CPU and PCIE , it may be very slow.

but its not a bad idea, we need someone to try this.

[TeKett]

PCIE 4.0 is 64 GB per second. From personal experience with waifu2x and making images that need 60+ GB ram, its not as slow as you think. People are biased because of gaming, cus that matters for framerate.

Edit: Im not sure how different this is, and how things get updated here, but someone over at huggingface are currently working on reducing the minumum vram to 700mb. Unless you want to dabble in the technowizarding thats going on in these threads.

[marcussacana]

I hope this feature became true as well, will be help to create larger resolution images than the usual limit for example

[VictorZakharov]

Video memory speed is 500GB/s and higher, for newer generations of video cards like 40xx. Regular memory speed (DDR4) is 50GB/s and higher. I would be fine to take 10x performance hit but finish without a crash.

Also it's possible that increasing memory amount will speed up the process, because it avoids having to shuffle things in/out of memory. Then buying 128GB of memory is cheap, on the order of 100 times cheaper than buying a video card with the same memory capacity.

[sinanisler]

i know there was a solution out there for torch that webui uses and it is possible to use that shared memory

but I don't have enough python or ml knowledge to code a PR here :)

lets hope someone does...

[aleimu]

+1

[arnebp]

+1

[sensty]

we really need this

[SoundGuy]

I support this request as well. I have 16GB shared but only 6GB GPU which makes it impossible to dreambooth

[Skybeat]

I support this as well, I have 6GB dedicated but 30 shared. I would be so happy to have this 30 for use as well and don't get all the errors because of low memory.

[RaphielHS]

+1 Dedicated 2.0GiB and 5.9GiB Shared

SD Works fine on 2Gigs tbh, its just slow and limited to 512x512

[Enderlook]

+1

I have 6 GB Dedicated and 16 GB Shared.

Being able to use 22 GB would be amazing to make large images.

[cch9897]

There are other ways to reduce VRAM usage, such as using Tiled Diffusion/Tiled VAE.

[Alluseri]

+1, I only have 8 GB VRAM and being able to use shared memory, even in exchange for slower generation, would be awesome.

[TeKett]

I think this has been implement(?) unless something else has changed, i have not checked the change logs. I recently got a new gpu, 4070 Ti, with more vram. Decided to try out some AI art again, and if i go over my vram it starts to use my shared memory rather then throwing an error, no issues capping out my 64gb ram. It is significantly slower, tho i can can go above like 2 GB with minimal slowdown.

On Wed, 28 Jun 2023 at 05:18, Alluseri @.***> wrote:

+1, I only have 8 GB VRAM and being able to use shared memory, even in exchange for slower generation, would be awesome.

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2550#issuecomment-1610624147, or unsubscribe https://github.com/notifications/unsubscribe-auth/APLFBP4BKK634HWAOPIHUXLXNOPBDANCNFSM6AAAAAAREZP3RM . You are receiving this because you commented.Message ID: @.***>

[eastinq]

I have an RTX 3070 8gb. Shared GPU of 16gb totally unused. When I try to load base SDXL, my dedicate GPU memory went up to 7.8gb of 8.0gb even before generating any images. No memory left to generate a single 1024x1024 image. I'm now using "set COMMANDLINE_ARGS= --xformers --medvram". I can now generate SDXL with using refiner too. GPU memory shoots up to 7.0 of 8.0 during generation but comes back down to 1.4 of 8.0 after finish. It would be nice to be able to use Shared GPU even if performance is slower. The trend is higher and higer resolution, so we need all available vram utilized.

[catboxanon]

I'm going to close this because this is what nvidia's latest drivers have been doing for quite some time now. And as expected, and as you can tell from the issue reports, it just makes things worse for the majority of users. https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/11063

[funkypickledpickle]

It appears that the latest Stable Diffusion webui uses shared memory. I have gtx 1660 that has 6gb of vram, and I'm using a model that weights around 7gb on the drive, although I'm not sure that it's loaded as is. No software was opened except a browser with 4 tabs. The generation speed is almost ~10 times slower than using --lowvram for the same model. On the screenshot you can see that the gpu is loaded up to 100% and almost 10 gb of shared memory is used.

[TeKett]

To sorta close down this issue tho, nvidia have added a toggle, both global and per application to turn this on and off. Its hot garbage tho for ai stuff since it spends too much time memory managing then letting the program do its thing.