text-generation-webui
text-generation-webui copied to clipboard
Error loading GLM-10b-chinese even with --trust-remote-code
Describe the bug
Traceback (most recent call last): File “/mnt/e/git/text-generation-webui/server.py”, line 85, in load_model_wrapper shared.model, shared.tokenizer = load_model(shared.model_name) File “/mnt/e/git/text-generation-webui/modules/models.py”, line 56, in load_model model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code) File “/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py”, line 441, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File “/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py”, line 919, in from_pretrained raise ValueError( ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.
Is there an existing issue for this?
- [X] I have searched the existing issues
Reproduction
python server.py --model THUDM_glm-10b-chinese --trust-remote-code
Warning: trust_remote_code is enabled. This is dangerous.
Loading THUDM_glm-10b-chinese... Traceback (most recent call last): File "/mnt/e/git/text-generation-webui/server.py", line 905, in
shared.model, shared.tokenizer = load_model(shared.model_name) File "/mnt/e/git/text-generation-webui/modules/models.py", line 56, in load_model model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code) File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 919, in from_pretrained raise ValueError( ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True
to remove this error.
Screenshot
No response
Logs
python server.py --model THUDM_glm-10b-chinese --trust-remote-code
> Warning: trust_remote_code is enabled. This is dangerous.
>
> Loading THUDM_glm-10b-chinese...
> Traceback (most recent call last):
> File "/mnt/e/git/text-generation-webui/server.py", line 905, in <module>
> shared.model, shared.tokenizer = load_model(shared.model_name)
> File "/mnt/e/git/text-generation-webui/modules/models.py", line 56, in load_model
> model = LoaderClass.from_pretrained(Path(f"{shared.args.model_dir}/{shared.model_name}"), low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if shared.args.bf16 else torch.float16, trust_remote_code=trust_remote_code)
> File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
> config, kwargs = AutoConfig.from_pretrained(
> File "/home/xxxxx/anaconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 919, in from_pretrained
> raise ValueError(
> ValueError: Loading models/THUDM_glm-10b-chinese requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error.
System Info
OS: WSL ubuntu 22.04
GPU: RTX 4090
This model has a completely different interface than chatglm. Even if you load it like this, it won't generate correctly.
I tried it as well and it goes OOM for me when loading. As said, it's a different model with different interface.
This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.