loading model errors

[ARCIST-AI]

Describe the bug

I recently reinstalled my os so i had to reload the webui and ive had some issues. Ive downloaded a couple of mixtral gguf models. When I load them the model loader box beneath the downloaded model box will switch to llama.cpp and i get a bunch of errors to do with llama.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

load a model

Screenshot

No response

Logs

^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'Llama'

14:41:48-159727 INFO     Loading "mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf"       
14:41:48-226431 INFO     llama.cpp weights detected:                            
                         "models/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf"        
14:41:48-230224 ERROR    Failed to load the model.                              
Traceback (most recent call last):
  File "/Users/alimac/textgen/modules/ui_model_menu.py", line 245, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(selected_model, loader)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alimac/textgen/modules/models.py", line 87, in load_model
    output = load_func_map[loader](model_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alimac/textgen/modules/models.py", line 250, in llamacpp_loader
    model, tokenizer = LlamaCppModel.from_pretrained(model_file)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alimac/textgen/modules/llamacpp_model.py", line 63, in from_pretrained
    Llama = llama_cpp_lib().Llama
            ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'Llama'

System Info

Hardware Overview:

  Model Name:	Mac mini
  Model Identifier:	Macmini7,1
  Processor Name:	Dual-Core Intel Core i7
  Processor Speed:	3 GHz
  Number of Processors:	1
  Total Number of Cores:	2
  L2 Cache (per Core):	256 KB
  L3 Cache:	4 MB
  Hyper-Threading Technology:	Enabled
  Memory:	16 GB
  System Firmware Version:	483.0.0.0.0
  OS Loader Version:	540.120.3~37
  SMC Version (system):	2.24f32
  Serial Number (system):	C07SR0Y3G1J2
  Hardware UUID:	D64EE4C5-FDAB-546D-8DA8-AAF69B29CB70
  Provisioning UDID:	D64EE4C5-FDAB-546D-8DA8-,AAF69B29CB70

Screenshot 2024-03-27 at 16.15.12.png

[MateuszUlan]

I have same issue using different model.

[crandev]

Same issue after installing a few moments ago on a new laptop. There is also another bug report thread about it: https://github.com/oobabooga/text-generation-webui/issues/5765

Happens with all models and model loaders.

[github-actions[bot]]

This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

[Melyns]

Same here.