Unkown, first time start errors.

[eddiehowell12]

Describe the bug

I'm unsure, I've used the One-time installer. And everytime I keep getting stuff that doesn't work.

I've done multiple restarts and reinstalls and something goes wrong everytime. This go arounds problem is in the Logs.

I'd really apreciate any support that can be offered, as using a one time installer multiple times is quite frustrating.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

No Idea, but the results are consistant until I change something. The something else is consistantly wrong.

Screenshot

No response

Logs

Gradio HTTP request redirected to localhost :)
bin C:\Users\edwar\Downloads\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
Loading mayaeary_pygmalion-6b_dev-4bit-128g...
Found the following quantized model: models\mayaeary_pygmalion-6b_dev-4bit-128g\pygmalion-6b_dev-4bit-128g.safetensors
Loading model ...
Traceback (most recent call last):
  File "C:\Users\edwar\Downloads\oobabooga_windows\text-generation-webui\server.py", line 914, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "C:\Users\edwar\Downloads\oobabooga_windows\text-generation-webui\modules\models.py", line 158, in load_model
    model = load_quantized(model_name)
  File "C:\Users\edwar\Downloads\oobabooga_windows\text-generation-webui\modules\GPTQ_loader.py", line 176, in load_quantized
    model = load_quant(str(path_to_model), str(pt_path), shared.args.wbits, shared.args.groupsize, kernel_switch_threshold=threshold)
  File "C:\Users\edwar\Downloads\oobabooga_windows\text-generation-webui\modules\GPTQ_loader.py", line 77, in _load_quant
    model.load_state_dict(safe_load(checkpoint), strict=False)
  File "C:\Users\edwar\Downloads\oobabooga_windows\installer_files\env\lib\site-packages\safetensors\torch.py", line 101, in load_file
    result[k] = f.get_tensor(k)
RuntimeError: shape '[50400]' is invalid for input of size 0

Done!
Press any key to continue . . .

System Info

Processor	12th Gen Intel(R) Core(TM) i3-12100F   3.30 GHz
Installed RAM	16.0 GB (15.8 GB usable)
System type	64-bit operating system, x64-based processor
Graphics Card   NVIDIA GeForce RTX 3050

[askmyteapot]

So from what i can tell, that model uses a totally different GPTQ version. You'll need to use https://github.com/mayaeary/GPTQ-for-LLaMa/tree/gptj-v2

EDIT: and it's unclear if that fork of GPTQ will work with ooba.

[xNul]

@eddiehowell12 You could also just try using a different model like https://huggingface.co/wcde/llama-7b-4bit-gr128 which definitely works with webui

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.