stable-diffusion-webui
stable-diffusion-webui copied to clipboard
AUTOMATIC1111
[Issue]: Half speed in generating images for same settings after update
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
Since today's update, image generation takes twice as long for the same input and settings as it did 2 days ago.
Steps to reproduce the problem
Update files.
What should have happened?
.
Commit where the problem happens
November 11 2022 7ba3923d5b494b7756d0b12f33acb3716d830b9a
What platforms do you use to access UI ?
Windows
What browsers do you use to access the UI ?
Google Chrome
Command Line Arguments
No response
Additional information, context and logs
No response
I haven't personally noticed any changes in speed. Have you tried reverting to the commit you were on before to intentionally compare it to the current build to see if there is actually a difference in generation speed?
I am getting the same problem. Was working just fine on unknown commit from about three weeks ago. Ran git pull and now it's running at half speed. Reverting to previous commits doesn't seem to fix the issue. It's still running at more or less exactly half speed.
I am running this on Ubuntu Server 22.04. Nvidia 3090.
Master branch 98947d173e3f1667eba29c904f681047dea9de90
Same problem using RTX 2080 8GB. almost 5-6 step per seconds one or two weeks ago with xformer enabled and medvram, now barely 3.
198a1ffcfc963a3d74674fad560e87dbebf7949f - Nov 1 Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:59<00:00, 8.38it/s] 512x512 Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:45<00:00, 2.61it/s] 1024x1024 HiresFix
98947d173e3f1667eba29c904f681047dea9de90 - Nov 12 Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:58<00:00, 8.43it/s] 512x512 Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:30<00:00, 2.66it/s] 1024x1024 HiresFix
98947d173e3f1667eba29c904f681047dea9de90 - Fresh Nov 12 / Nov 13 Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:55<00:00, 8.64it/s] 512x512 Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:03<00:00, 2.76it/s] 1024x1024 HiresFix
RTX 3070, Xformers, deepdanbooru
All settings default, Sampler changed to Euler
I couldnt replicate this..
I am getting the same problem. Was working just fine on unknown commit from about three weeks ago. Ran git pull and now it's running at half speed. Reverting to previous commits doesn't seem to fix the issue. It's still running at more or less exactly half speed.
I am running this on Ubuntu Server 22.04. Nvidia 3090.
Master branch 98947d1
@mahjongchess
run git reflog and see your previous commit versions you were using.
everyone mind installing a fresh webui to see what went down? (no need to delete previous) Also it would be very helpful to narrow down the commit causing this for the devs
Did any of you recently update your Nvidia drivers? I tend to stick with known good drivers until a stable update without major issues reported by users is released (right now still using 511.79), but I have heard that the latest Nvidia drivers were bungled somehow and have been causing problems with ML-related stuff.
Did any of you recently update your Nvidia drivers? I tend to stick with known good drivers until a stable update without major issues reported by users is released (right now still using 511.79), but I have heard that the latest Nvidia drivers were bungled somehow and have been causing problems with ML-related stuff.
No (sorry if you read previous comment), I downloaded 522.30 nvidia studio driver but didn't install it. At present, running 512.95 Geforce game ready driver from 24/05/22
198a1ffcfc963a3d74674fad560e87dbebf7949f - Nov 1
Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:59<00:00, 8.38it/s] 512x512
Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:45<00:00, 2.61it/s] 1024x1024 HiresFix
98947d173e3f1667eba29c904f681047dea9de90 - Nov 12
Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:58<00:00, 8.43it/s] 512x512
Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:30<00:00, 2.66it/s] 1024x1024 HiresFix
98947d173e3f1667eba29c904f681047dea9de90 - Fresh Nov 12 / Nov 13
Total progress: 100%|██████████████████████████████████████████████████████████████| 1000/1000 [01:55<00:00, 8.64it/s] 512x512
Total progress: 100%|██████████████████████████████████████████████████████████████| 2000/2000 [12:03<00:00, 2.76it/s] 1024x1024 HiresFix
RTX 3070, Xformers, deepdanbooru
All settings default, Sampler changed to Euler
i updated mine and have noticed a similar decrease. 2080 super m went from 6-7 it/s to ~3 it/s. This was using GRD not studio drivers. I might test with studio drivers and then post back here.
I couldnt replicate this..
I am getting the same problem. Was working just fine on unknown commit from about three weeks ago. Ran git pull and now it's running at half speed. Reverting to previous commits doesn't seem to fix the issue. It's still running at more or less exactly half speed. I am running this on Ubuntu Server 22.04. Nvidia 3090. Master branch 98947d1
@mahjongchess run
git reflogand see your previous commit versions you were using.everyone mind installing a fresh webui to see what went down? (no need to delete previous) Also it would be very helpful to narrow down the commit causing this for the devs
I did a fresh install. And got back original speed Data shape for DDIM sampling is (1, 4, 64, 64), eta 0.0 Running DDIM Sampling with 20 timesteps DDIM Sampler: 100%|████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.89it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.15it/s] Data shape for DDIM sampling is (1, 4, 64, 64), eta 0.0 Running DDIM Sampling with 50 timesteps DDIM Sampler: 100%|████████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00, 6.19it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:07<00:00, 6.28it/s] 100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:07<00:00, 6.36it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00, 6.18it/s] 100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00, 6.20it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:07<00:00, 6.29it/s] 100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00, 5.63it/s] Total progress: 100%|
Activating preveiew every 5 steps results in 10% decreased speed :
██████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00, 5.72it/s] Data shape for DDIM sampling is (1, 4, 64, 64), eta 0 Running DDIM Sampling with 50 timesteps DDIM Sampler: 100%|████████████████████████████████████████████████████████████████████| 50/50 [00:09<00:00, 5.34it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:09<00:00, 5.38it/s]
I couldnt replicate this..
I am getting the same problem. Was working just fine on unknown commit from about three weeks ago. Ran git pull and now it's running at half speed. Reverting to previous commits doesn't seem to fix the issue. It's still running at more or less exactly half speed. I am running this on Ubuntu Server 22.04. Nvidia 3090. Master branch 98947d1
@mahjongchess run
git reflogand see your previous commit versions you were using.
Thanks, I should have thought of that. Looks like cd5eafaf03a25d2b0e35154666947b9489078af9 is what I was using and was working quite well. I'll pull cd5eafaf03a25d2b0e35154666947b9489078af9 into a new directory and see if it works.
OK, so I'm still getting the same results. The generation is absolutely slower and I can't seem to figure out why, but cloning the commit that worked into a new dir is also causing the problem. Maybe it's something that gets installed automatically by webui.sh.
This is the output of nvidia-smi:
I have a few screenshots of it running at 100% when generating a large batch of images somewhere that I can't currently find, but I'm absolutely sure it always ran at 100% (or close to it) when generating images.
Found the older screenshot of nvidia-smi. Looks like from Nov 1.
Look at the wattage used.
A batch of 100, 4 at a time of 512x768, 32 steps, eular_a would take roughly half an hour before, and now it's taking roughly an hour.
Do we have a resolution to this yet? For the past few days now I've had the same problem, really slow generation. It doesn't even seem to be taxing the GPU anymore, temps sit at idle and not much happens. Images that took 40 seconds now take a few minutes. This was seemingly after updating, but even after making another SD folder using and older version, its still slow now.
I tried pulling cd5eafaf03a25d2b0e35154666947b9489078af9 again into a new directory. Everything's working at full speed now. I'm honestly not sure why it didn't when I tried the other day. Maybe I wasn't in that directory when I ran webui.sh.
@RyuHayabusa77 no, not really
@Ehplodor said it it was --medvram
probably --medvram and --xformers at the same time.
I used it too, as some time ago TheLastBen noted it increased performance, and I did see a slight speed increase. Eventually maybe the updates broke that specific speed optimization
I tried pulling cd5eafa again into a new directory. Everything's working at full speed now. I'm honestly not sure why it didn't when I tried the other day. Maybe I wasn't in that directory when I ran webui.sh.
Thanks very much! That one seems to have solved the problem! Getting normal speeds and GPU utilization again!
(and just to clarify, I wasn't using --medvram or --xformers. I figured my 1080Ti's 11GB of VRAM was enough and didn't require the --medvram)
Can also confirm I'm not using --medvram or --xformers as the 3090 doesn't need it. I'm only using --listen as this is on a server. cd5eafaf03a25d2b0e35154666947b9489078af9 continues to run beautifully.
