Can't run starchat: fails with `AttributeError: module 'global_vars' has no attribute 'gen_config'`

[nathanielastudillo]

Trying to run starchat and getting an error. The model downloaded but when I click "Confirm", I just get an error.

Update: Also getting this error when trying to download models. Might be an env issue, going to nuke my conda env and try again.

[nathanielastudillo]

Wiped my conda env, tried to install another model (alpacoom-7b), getting the same error:

AttributeError: module 'global_vars' has no attribute 'gen_config'

[kdubba]

There might be some other error before this. Maybe the model could not be loaded because of memory limits. Could you check the error log?

[sandeepraizada]

Here is a part of the error detail before the the error. I have made a few changes to the file to run it in Google Colab, bit the error is the same as reported earlier.

File "/usr/local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2969, in _load_pretrained_model raise ValueError( ValueError: The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format. Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/gradio/routes.py", line 427, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/content/LLM-As-Chatbot/app.py", line 218, in move_to_third_view gen_config = global_vars.gen_config AttributeError: module 'global_vars' has no attribute 'gen_config'

[deep-diver]

what kind of changes have you made?

[deep-diver]

also, notice that I make updates very frequently, so please git pull whenever you try

[sandeepraizada]

I have used Google colab so

1. used the conda install there
2. changed the app.py and set shared= True to get a public link. however i was not using a GPU.. now with a GPU... here is the second error message File "/usr/local/lib/python3.10/site-packages/peft/peft_model.py", line 167, in from_pretrained PeftConfig.from_pretrained(model_id, subfolder=kwargs.get("subfolder", None), **kwargs).peft_type File "/usr/local/lib/python3.10/site-packages/peft/utils/config.py", line 110, in from_pretrained raise ValueError(f"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'") ValueError: Can't find 'adapter_config.json' at 'LLMs/Alpaca-LoRA-7B-elina' Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/gradio/routes.py", line 427, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/content/LLM-As-Chatbot/app.py", line 218, in move_to_third_view gen_config = global_vars.gen_config AttributeError: module 'global_vars' has no attribute 'gen_config'

[deep-diver]

I would not recommend to use Colab since the connection is not stable. This error is due to the unstable connection that will eventually be terminated shortly (I don't know why)

[sandeepraizada]

It works with t5-vicuna though.

[deep-diver]

yeah since t5-vicuna is the smallest model which don't take too much time to load up

[sandeepraizada]

thought as much to try with Vicuna being a smaller model. thanks for your response!

[deep-diver]

T5-Vicuna is 3B model, hence :)

[bitsnaps]

I got the same error when running on Gitpod (the 16Gb container), trying to run on CPU mode, the only changes:

# for torch
pip install torch==2.0.0+cpu torchvision==0.15.1+cpu -f https://download.pytorch.org/whl/torch_stable.html

# for auto-gptq
BUILD_CUDA_EXT=0 pip install auto-gptq 

using Python v3.11.1.

[gavi]

In my case the download_complete was getting the model_name and model_base with html tags, I cleaned up with the following

    print(f"model_name: {model_name}")
    print(f"model_base: {model_base}")
    model_name = model_name.replace("<h2>","").replace("</h2>","").strip()
    model_base = clean_up(model_base)
    model_ckpt = clean_up(model_ckpt)
    model_gptq = clean_up(model_gptq)
    

The clean_up function is below

def clean_up(model_base):
    pattern = r":\s*(\w+/[\w-]+)"
    match = re.search(pattern, model_base)

    result = match.group(1) if match else model_base
    return result