[Bug]: Please do some optimisation for new Hire.Fix (specially to LDSR) on 8GB VRAM because it fails often with out of memory and eventual results still are disappointing

[Centurion-Rome]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Using "old Hires.Fix" I had no problems and could generate images like 1600x1088 or 1536x1152 and "Hires.Fix" with no problems.

With new Hires.fix (no matter which upscaler I choose) I am getting "CUDA out of memory" !

Example: 768x576 => Hires.Fix Upscaler "none" Upscale by 2: (to 1536x1152) => out of memory

Most interesting will be "LDSR" :)

Steps to reproduce the problem

1. Use RTX 2080 Super with 8GB VRAM
2. set txt2img with: 768x576 => Hires.Fix Upscaler "none" Upscale by 2: (to 1536x1152) => out of memory
3. Click generate See error: "RuntimeError: CUDA out of memory. Tried to allocate 864.00 MiB (GPU 0; 8.00 GiB total capacity; 5.70 GiB already allocated; 0 bytes free; 6.59 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"

Happens after DDIM was successfully ready: " Downsampling from [768, 512] to [384, 256] Plotting: Switched to EMA weights Sampling with eta = 1.0; steps: 50 Data shape for DDIM sampling is (1, 3, 256, 384), eta 1.0 Running DDIM Sampling with 50 timesteps DDIM Sampler: 100%|████████████████████████████████████████████████████████████████████| 50/50 [00:31<00:00, 1.60it/s] Plotting: Restored training weights Error completing request.....CUDA error "

What should have happened?

Image should be generated without error, please fix and bring also "old Hires.fix" as additional option (it was working just fine in some cases)

If I manage to upscale I a little bit (without out of memory) the results are disappointing in comparison to the old version! So in current state it can not be used in production as IMHO is a downgrade (not working or still doubled objects/parts).

I would be happy to see old one again as option.

Commit where the problem happens

3fa0cd23ddacdb348e672c98c07d0d08c6f61c32

What platforms do you use to access UI ?

No response

What browsers do you use to access the UI ?

No response

Command Line Arguments

--xformers

Additional information, context and logs

No response

[Centurion-Rome]

Current implementation is useless for me because of to resource hungry processes.

[metzo007]

same here. Hires.fix went from 10 sec to 3 min proccecing time often ending in out of memory errors

[metzo007]

FYI. There is a workaround. Generate your images in low res, and then a batch img2img in the high res. ;-)

[tchesket]

Same issue here. Hires fix giving CUDA out of memory a lot now (rtx 3070 8gb), sometimes it works fine doing 1024x768 even but usually not, even 768x512 usually fails.

[Centurion-Rome]

FYI. There is a workaround. Generate your images in low res, and then a batch img2img in the high res. ;-) I know, but That is not a fix for Hires.Fix

[tchesket]

Yeah but that doesn't really help my use case, I'm running into this issue using the multi-subject-render extension/script which uses Hirex.Fix during processing, that's where I keep running out of VRAM

[Centurion-Rome]

Please close this ticket and do https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/6725