silvestron

Is it okay to use llama for this model? When I tried to set model type to mpt I got his error: ``` ERROR:Unknown pre-quantized model type specified. Only 'llama',...

@jpturcotte Can you run `git show` inside the text-generation-webui folder to see what commit you're on? How do you get to run it though? I have to manually specify the...

@NickWithBotronics `"use_cache": false` in the config file, I haven't touched it. How much does the webui respect the config file though? It doesn't seem to care about the model type,...

I replaced `git pull` with `git checkout 85238de` in webui.py, but it looks like going back to older commit breaks things. Maybe running a clean install with that commit would...

I'd also add that on a working, up to date installation, I tried using llama, gptj, and opt as model type and gave the same results.

Are we talking about the 4bit model? That doesn't work if you don't specify a model. #1894 I get the same error if I don't give it a model type.

That actually made the token generation faster, however the initialization time, that on my hardware takes 9 minutes didn't change. The config has `"init_device": "cpu"` and the console says you...

@jpturcotte All good, I couldn't do much with this model anyway without much of VRAM anyway. I guess multi-GPU is going to be the only way to run models that...

I guess I can reinstall, but I wonder what broke my current installation.

I did a clean install but that didn't solve the problem.