Add a load model button instead of loading on model selection

[LaaZa]

Automatically loading models when they are selected is a pain. You can't change the settings before the loading either succeeds or fails and this can take a long time. The workaround is unintuitive and you still have to wait.

This PR solves this by adding a Load the model button and removes the loading from the model selection event.

[oobabooga]

I was thinking of a "Stop loading" button instead for cases where you regret loading the model, but I don't know if it's doable

[LaaZa]

I don't see the reason why loading should be done when you select the model, just causes more problems.

You might also want to load the settings for the model just to see what they were. And if it is not loaded automatically you can also change them first.

[Ph0rk0z]

It is really bad when you have a 4bit and an FP16 in the same folder and it starts loading shards. :(

[LoopControl]

+1 on this.

As @Ph0rk0z mentioned, it'd be great to not have it auto-load the full-precision version every time I want to set 4bit mode and play with groupsize settings and such.

[LaaZa]

@oobabooga think you can merge this soon or give a reason why not? I'm going crazy with how many times the current loading system has f'ed me over. We can add your stop button too when you figure out how it would work, but I think this would be useful for people right now.

[oobabooga]

i have tried adding a stop button today but it didn't work, I think for the same reason why the "Stop" button doesn't work when the model is outputting 1 token every minute. Namely, you can't kill a Python thread

The whole purpose of loading the model specific settings was to seamlessly switch between models, and adding a new button click is a step in the opposite direction. It would also confuse many users. Maybe add a checkbox called "automatically load selected models?" that can be disabled?

[LaaZa]

i have tried adding a stop button today but it didn't work, I think for the same reason why the "Stop" button doesn't work when the model is outputting 1 token every minute. Namely, you can't kill a Python thread

The whole purpose of loading the model specific settings was to seamlessly switch between models, and adding a new button click is a step in the opposite direction. It would also confuse many users. Maybe add a checkbox called "automatically load selected models?" that can be disabled?

If loading a model took like a second then maybe, but they can take minutes depending on the situation. Also it loads the settings you had saved(or premade) you have to load the model again to change them. Sometimes you have settings that just kill the whole process and you can't easily change them next time. Come on, this only makes sense in SD webui because it doesn't really have model specific settings and they are fairly quick to load. I have had so many situations where the selection opened again and I hit some other model from the list where it was faster to kill the thing instead of waiting for both models to load again.

Also loading models will never be seamless for everybody because they have very different setups and there is a need to change some settings and many times they have to try different ones to find the correct solution. And while not recommended, some people have different types of models in one modeldir where only the settings for that model(dir) will decide which one is loaded.

[LoopControl]

The whole purpose of loading the model specific settings was to seamlessly switch between models

The purpose of the model-settings (very convenient by the way) is so we don’t have to remember and punch them in every time.

I think you’re over estimating the amount of people who switch models that frequently and frankly, one button to load a model isn’t that much work.

What is an absolute pain is to try to select a model from a list and misclick or not have saved settings and have 3+ minutes of maxed out CPU/harddrive grinding just to GPU memory overflows instead of the 15 second load in GPTQ mode or whatever is the correct setting.

Also if you’re not sitting at the same machine that webui is running on, you have no way to kill a multi-minute auto-load.

If these models were a few MB, it’d be a different story but auto-loading multi-GB models unintentionally or with wrong settings is terrible.

[LaaZa]

Also I would like to add that the checkbox is too much I think, I would hazard a guess that there are much less people "inconvenienced" by having to press a button than there are for accidental model loads and the unintuitive way you have to change the settings because you can only do that after and then reload.

I played around with a checkbox option but I could not for the life of me get it working sensibly, that is probably partly due to my inexperience with gradio, but also partly due to how the UI is set up. Eitherway I do not see the value in that option to warrant complicating the code.

You can make the Load button stand out more by adding variant='primary' but I think you can do that yourself quickly unless there is something more substantial that needs to be changed.

[Ph0rk0z]

Can't we have both. So a check box if you want the model to load without intervention and a load button if you disable the check box.

[LaaZa]

Can't we have both. So a check box if you want the model to load without intervention and a load button if you disable the check box.

If ooba wants to do that, but I could not get it to work and I think it requires annoying amount of changes to the code for it to work well. I feel it is simply better not to have too many options, I personally don't see any benefits for the automatic loading.

[oobabooga]

I have added this new checkbox that may make everyone happy

@LaaZa @Ph0rk0z @LoopControl can you test if it works as expected?

[LaaZa]

@oobabooga I guess that works. I would prefer it to be off by default though and I mean without changing the settings, I think the manual loading is more beneficial to most people than automatic and they may not realize what that option is.

"You can change this default in settings.json" I fear is gonna make them come asking where that is since it does not exist by default.

Also, while not for this PR, I think the settings should be yaml or something instead of json since it is manually edited.

I actually think this should be sort of "hidden" advanced option. Not in the UI at all. No one is going to come asking how to make the model load automatically on selection, but they will ask where that settings json is. But whatever, if you want to be difficult and deal with all the issues. I just want the models not to load automatically.

[LoopControl]

Just tested - works great thanks!

[Ph0rk0z]

Yup.. will just change my config file to keep not loading the model by default.

[LoopControl]

Yup.. will just change my config file to keep not loading the model by default.

Hadn’t realized it was customizable via config file too - thanks for the tip!