BetaDoggo

Add `--xformers --opt-split-attention --use-cpu interrogate` to your command-line arguments in your webui-user.bat file. With a 2060 you should be able to generate images up to at least 1024x1024. Also make...

It looks like the issue is that you're trying to load in 4bit mode but the webui doesn't detect the model type so it fails. Try adding `--gptq-model-type LLaMa` to...

8bit on gpu via bitsandbytes is known to be slower than fp16. On a 3090 you should be able to fit the full fp16 version of the model so there...

Classic windows defender. I've gotten similar alerts for other pytorch models. Almost definitely a false positive.

Did you move the tokenizer files into the model folder after converting? I made that mistake originally and had the same error.

I believe you also need `tokenizer.model`. It might be in the folder of the original unconverted model rather than the one created by the conversion script.

Requirements vary greatly between models, there's a list of some examples of vram/ram usage for different models on the [wiki](https://github.com/oobabooga/text-generation-webui/blob/main/docs/System-requirements.md). This list doesn't include 4bit models but they should use...

There's [this project](https://github.com/EmpireMediaScience/A1111-Web-UI-Installer) which kind of does that. The normal webui installation is only like 2 steps though.

I've also noticed a loss in performance after this commit that I didn't see in the original PR. The generation process seems to freeze for a few seconds at the...

I went from a max of 3 to 15 at 512x512 on a 6GB RTX 2060. Really impressive!