[Bug]: Upscaling process appears to be happening twice

[DivineOmega]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

When upscaling in the extras tab, I have noticed the upscaling process taking much longer than usual.

When looking at the command line, it appears to be running the upscaling process twice, even when the second upscaler is set to 'None'.

~I have identified this is caused by the commit https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/29f758afe9c09834a1cd2dac3f5dd6fbf7dfeaa7. When checking a commit out before this one, the upscaler works as before.~

Not sure the above is actually the cause after further testing, but the issue still remains.

Steps to reproduce the problem

1. Go to the extras tab -> single image
2. Upload/paste in an image
3. Select 'R-ESRGAN 4x+' as 1st upscaler
4. Select 'None' as 2nd upscaler
5. Notice the longer time to upscale and the output in the terminal showing the upscaling process occurring twice, as below.

        Tile 1/9
        Tile 2/9
        Tile 3/9
        Tile 4/9
        Tile 5/9
        Tile 6/9
        Tile 7/9
        Tile 8/9
        Tile 9/9
        Tile 1/121
        Tile 2/121
        Tile 3/121
        Tile 4/121
        Tile 5/121
        Tile 6/121
        Tile 7/121
        Tile 8/121
        Tile 9/121
[...]

What should have happened?

Upscaling process should only happen once.

Commit where the problem happens

172c4bc09f0866e7dd114068ebe0f9abfe79ef33

What platforms do you use to access UI ?

Linux

What browsers do you use to access the UI ?

Google Chrome / Brave

Command Line Arguments

No response

Additional information, context and logs

No response

[aliencaocao]

I dont see the issue, tried with both ESRGAN 4x and R-ESRGAN 4x anime + 6B. I am on Windows, latest commit

[Bill4321]

I get the same duplication problem with LDSR. I have it set up for a 512x512 image and a resize of 2. When I run it I get this..

Loading model from C:\Users\stable-diffusion-webui\models\LDSR\model.ckpt LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 113.62 M params. Keeping EMAs of 308. making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 3, 64, 64) = 12288 dimensions. making attention of type 'vanilla' with 512 in_channels Downsampling from [512, 512] to [256, 256] Plotting: Switched to EMA weights Sampling with eta = 1.0; steps: 100 Data shape for DDIM sampling is (1, 3, 256, 256), eta 1.0 Running DDIM Sampling with 100 timesteps DDIM Sampler: 100%|██████████████████████████████████████████████████████████████████| 100/100 [01:27<00:00, 1.14it/s] Plotting: Restored training weights Loading model from C:\Users\stable-diffusion-webui\models\LDSR\model.ckpt LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 113.62 M params. Keeping EMAs of 308. making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 3, 64, 64) = 12288 dimensions. making attention of type 'vanilla' with 512 in_channels Downsampling from [1024, 1024] to [512, 512] Plotting: Switched to EMA weights Sampling with eta = 1.0; steps: 100 Data shape for DDIM sampling is (1, 3, 512, 512), eta 1.0 Running DDIM Sampling with 100 timesteps DDIM Sampler: 100%|██████████████████████████████████████████████████████████████████| 100/100 [08:07<00:00, 4.87s/it] Plotting: Restored training weights

The end result is an image which has been correctly upscaled from 512x512 to 1024x1024 but now taking ten minutes or so.

I have upscaler 2 set to none with a visibility of 0.

I have tried rolling back 24 hours to get things back to as they were but no dice.

Hoping this can be fixed as I use LDSR one heck of a lot :-)

[ClashSAN]

could this be related https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/3976#issuecomment-1300079504

[r5v]

Noticed the same with LDSR. When it loads the LDSR model for the second time, the sampling process is about four times slower and it's probably because of different parameters of the scaling process. Please compare sampling parameters above.

[arktronic]

could this be related #3976 (comment)

Looks like it. I can confirm that the issue happens on e359268be9936db1c16c78adf544d622d33d1bfb when doing 4x upscaling, but not on bb21a4cb35986d95ec63ace48ce13b75a776f5a5.

When the issue occurs, this is what I see with SwinIR_4x:

SwinIR tiles: 100%|██████████| 9/9 [00:05<00:00,  1.55it/s]
SwinIR tiles: 100%|██████████| 144/144 [01:30<00:00,  1.59it/s]

[r5v]

Just did a quick test, and it seems that reverting this particular change works in my case. No double upscaling anymore. I know it's a dirty solution, but maybe it will help someone for now.

[Bill4321]

That works for me, thanks @r5v !!

[merson777]

@r5v

3:40 => 18 seconds - thank you

[victorca25]

Try https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/4226 to see if it fixes the issue

[CaptainValor]

Can confirm this is also happening for me. Please fix! LDSR upscaling is practically unusable at the moment. 😬

[victorca25]

@CaptainValor did you try the fix?

[arktronic]

@victorca25 testing e8ed4f4c55300417e52e04516246d60220ea32b3, the issue appears to be fixed.

[victorca25]

Nice @arktronic! From the discussion in there I'll do one more modification and make the PR ready to be merged if all looks good

[ClashSAN]

fixed