Much longer generation time on Hires. fix

[SeBL4RD]

same problem as above. But with the hires Fix On the hires. fix, even with very low resolutions (1152x896 x1,3), with a LoRA, after 50% (when enlarging), generation becomes extremely slow. With the same seed and settings, if I don't stop generation I get 16s/it instead of ~3. Once again, stopping/restarting the generation sometimes makes things run smoothly.

https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/1634