[Bug]: Cached checkpoints do not load when using separate vae files

[Kilvoctu]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

When using "Checkpoints to cache in RAM" feature with "SD VAE" set to auto, any cached checkpoints that have an associated VAE file will not load when swapping back to them. It will only load the weights from the VAE instead.

I've provided a short video demonstrating the issue. In it, you can see my WD checkpoint does not load properly and will use the weights from the previous checkpoint: https://streamable.com/h7dzd8

Steps to reproduce the problem

1. Go to settings
2. Set "Checkpoints to cache in RAM" to 1 or more
3. "SD VAE" set to anything that isn't none
4. Load a checkpoint that has a separate vae (it's now cached in RAM)
5. Swap to any other checkpoint
6. Swap back to the previous checkpoint

What should have happened?

The weights for the checkpoint should have been loaded, then it can load any VAE weights too.

Commit where the problem happens

30b1bcc64e67ad50c5d3af3a6fe1bd1e9553f34e

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

--xformers --opt-channelslast --listen --api --autolaunch --enable-insecure-extension-access

Additional information, context and logs

If "SD VAE" set to any specific VAE file, then no checkpoints will load properly when within the cache limit.

For example: I set my "SD VAE" to my WD VAE file Then set my "Checkpoints to cache in RAM" to 3 Imagine my checkpoints list is only these three checkpoints: [4470c325], [be7ddafc], and [ccf3615f] I launch the application, and it loads [4470c325] automatically, with the VAE I then load [be7ddafc], then [ccf3615f] The checkpoint is now stuck on [ccf3615f]

[aliencaocao]

Same issue here. I can toggle in webui, console also prints loading weight from cache, but using 2 very different models, the generated image is the same as if the model were not swapped. The webui erroneously displays the image as generated by the new model, when it is not.

This issue happens no matter what I set my VAE to be - specific file, auto or none

Steps to reproduce:

1. have 2 SD checkpoints and settings cache in ram set to 2
2. On start up, there will be 1 loaded. Generate using that gives expected results
3. Switch to the other one using webui, generate, also gives expected results
4. Switch back to the first one (the one loaded on start up), generate, still gives the same result as the 2nd one (loaded in step 3). This is bug.

I can confirm at https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/99043f33606d3057f83ea52a403e10cd29d1f7e7 this bug exists already. It may be introduced even earlier than this commit, but I have no time to test since the previous few commits have error in merging which cause it to not even launch.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/4283/ This fixes for me

[Kilvoctu]

This was fixed with e666220ee458ae1e80a2ba12c64a0da9d68f20a2