Improve VAE selector

[R-N]

There was still some issue or room for improvements in #3986, and some of them are just discovered.

1. The currently loaded VAE from auto isn't reflected in the settings dropdown nor displayed anywhere (other than the output log). The dropdown stays "auto". I'm not sure what to do about this.
2. The base VAE is stored in RAM when you load a separate VAE (for preparation when you select None). I guess this should be a setting (whether you want to store it in RAM, in file, or not at all so selecting None will require reloading checkpoint).
3. restore_base_vae was actually never called (you can see I actually called it wrong but error was never thrown), so "None" VAE option was bugged.
4. Base VAE caching was done after loading selected VAE. Should be before

So yeah here's some updates:

1. Fix "None" VAE option, both the caching and the usage (previously never called). Used for #3866
2. Use relative path as dropdown entries, similar to checkpoint dropdown
3. Added more fallback for auto (It will now search for similarly relatively located VAE in VAE folder. (e.g. you have Stable-diffusion/mymodel/mymodel.ckpt it will also try to fallback to VAE/mymodel/mymodel.vae.pt etc)
4. Added VAE caching options: ram, file, and none. "ram" means caching in memory. "file" will cache to ./tmp/base.vae.pt. "none" means no caching at all, so when you change to "None" VAE it will reload the checkpoint. Previously, caching was always done in memory (so that's around +300mb RAM usage I guess?). The default is now "file".
5. Added option to treat the VAE selector as default fallback. Priority will then become: (1) vae-path arg, (2) similar VAE as checkpoint (e.g. "beside" checkpoint, same path but .vae.pt ext), (3) selected VAE. Ref: #3655 and a comment
6. Show what VAE is used in auto by refreshing VAE dropdown. This includes change in how the setting tab works so that an option (settings item) can refresh other options when it's changed (after onchange).

2 rounds of test just like before:

Prompt:

ganyu \(genshin impact\),
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1, Size: 512x512, Model hash: 925997e9, Eta: 0.0022321428125, Clip skip: 2, ENSD: 31337

Order (2 rounds):

1. auto (final_pruned.vae.pt) (animevae)
2. wd_kl-f8-anime2
3. sd_1.5_mse
4. None

First round (for comparison between VAEs used, they should be different even just a little)

animevae: wd_kl-f8-anime2: sd_1.5_mse: None:

For comparison between rounds (grouped by VAEs, each having two images which should be the same):

animevae

wd_kl-f8-anime2

sd_1.5_mse

None

Another comparison test

I'm still not convinced so I run another test alternating between animevae and none. Same prompts. Order:

1. auto (animevae)
2. None

First round (for comparison between VAEs used, they should be different even just a little)

animevae: None:

animevae

None

Comparison between this one's None and the previous one (from the test with 4 options tried)

Current: Previous:

Embedding training test

Embedding training after switching to None (with default "file" caching). Now I'm finally sure the "None" option works. NAI model. "auto" resolves to animevae

Parameters

VAE "auto", select "None", then train

VAE "None", select "auto", then train

[Thomas-MMJ]

perhaps rename 'auto' to 'auto (something.vae.pt) where something is the name of the vae that was loaded.

[R-N]

perhaps rename 'auto' to 'auto (something.vae.pt) where something is the name of the vae that was loaded.

Yes, that's what update no. 6 does (by refreshing, because gradio doesnt allow this easily). Well, almost, because it shows what auto would load if you were to choose it (maybe it should also be refreshed after loading?).

But yeah I guess I wasn't being clear enough. I'll add pictures later.

[Omegastick]

I've been running with this branch merged locally for a while now, and I haven't had any noticeable issues.