text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Error loading GLM-10b-chinese even with --trust-remote-code

Open anitman opened this issue 1 year ago • 2 comments

Describe the bug

Traceback (most recent call last): File “/mnt/e/git/text-generation-webui/server.py”, line 85, in load_model_wrapper shared.model, shared.tokenizer = load_model(shared.model_name) File “/mnt/e/git/text-generation-webui/modules/models.py”, line 56, in load_model model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code) File “/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py”, line 441, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File “/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py”, line 919, in from_pretrained raise ValueError( ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.

Is there an existing issue for this?

  • [X] I have searched the existing issues

Reproduction

python server.py --model THUDM_glm-10b-chinese --trust-remote-code

Warning: trust_remote_code is enabled. This is dangerous.

Loading THUDM_glm-10b-chinese... Traceback (most recent call last): File "/mnt/e/git/text-generation-webui/server.py", line 905, in shared.model, shared.tokenizer = load_model(shared.model_name) File "/mnt/e/git/text-generation-webui/modules/models.py", line 56, in load_model model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code) File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 919, in from_pretrained raise ValueError( ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.

Screenshot

No response

Logs

python server.py --model THUDM_glm-10b-chinese --trust-remote-code

> Warning: trust_remote_code is enabled. This is dangerous.
> 
> Loading THUDM_glm-10b-chinese...
> Traceback (most recent call last):
>   File "/mnt/e/git/text-generation-webui/server.py", line 905, in <module>
>     shared.model, shared.tokenizer = load_model(shared.model_name)
>   File "/mnt/e/git/text-generation-webui/modules/models.py", line 56, in load_model
>     model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code)
>   File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
>     config, kwargs = AutoConfig.from_pretrained(
>   File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 919, in from_pretrained
>     raise ValueError(
> ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error.

System Info

OS: WSL ubuntu 22.04
GPU: RTX 4090

anitman avatar Apr 17 '23 09:04 anitman

This model has a completely different interface than chatglm. Even if you load it like this, it won't generate correctly.

sgsdxzy avatar Apr 17 '23 09:04 sgsdxzy

I tried it as well and it goes OOM for me when loading. As said, it's a different model with different interface.

Ph0rk0z avatar Apr 17 '23 12:04 Ph0rk0z

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.

github-actions[bot] avatar May 17 '23 23:05 github-actions[bot]