[Feature Request]: Option to unload VAE for textual embedding training

[R-N]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Textual embedding training goes bad when VAE is loaded. However unloading VAE requires webui restart, and if the VAE is named the same as the model with .vae.pt extension, it's still automatically loaded, so it has to be moved or renamed. It's inconvenient. It'd be really nice if there's a feature to unload and load VAE without having to restart the webui.

Additionally, it'd be better if in training tab it says whether VAE is loaded or not, with a shortcut to unload or a hint to unload from settings. A confirmation when trying to train embedding with VAE loaded would be even better, but idk if gradio supports it.

Proposed workflow

1. Go to settings
2. Press unload VAE button
3. Train embedding
4. Go to settings again
5. Press load VAE button

Additional information

Related:

• #2464 , but it was a question, not a feature request
• #3302 , so even for the hypernetwork previously, it wasn't really unloaded, but moved to RAM? Are they still in effect while in RAM?

[R-N]

Found out that the None option is still bugged in #3986 . Will PR fix later with other stuff

[R-N]

Solved #4841

[ddpasa]

Hello @R-N , do you think this is working now? If I try to train a textual inversion embedding with any model other than v1-5-pruned-emaonly, I get crazy RGB artifacts in the images during training. The resulting TI is also garbage. This sounds like the VAE is not being unloaded correctly.

I added sd_vae to quicksettings and selected None before training textual inversion. It makes no difference compated to setting it to automatic.

I'm using: version: v1.6.0 •  python: 3.10.12  •  torch: 2.0.1+cu118  •  xformers: 0.0.21  •  gradio: 3.41.2

with DucHaitenJourney5 model from Civitai: https://civitai.com/models/20261/duchaitenjourney