[BUG/Help] 已经下载chatglm2-6b模型，但是python web_demo.py抛出路径错误

[mainxx]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

如果是： tokenizer = AutoTokenizer.from_pretrained("chatglm2-6b/", trust_remote_code=True) model = AutoModel.from_pretrained("chatglm2-6b/", trust_remote_code=True, device='cuda') 则报错： PS C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main> python web_demo.py Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Traceback (most recent call last): File "C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main\web_demo.py", line 5, in <module> tokenizer = AutoTokenizer.from_pretrained("chatglm2-6b/", trust_remote_code=True) File "C:\Users\joven\miniconda3\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 663, in from_pretrained tokenizer_class = get_class_from_dynamic_module( File "C:\Users\joven\miniconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 399, in get_class_from_dynamic_module return get_class_in_module(class_name, final_module.replace(".py", "")) File "C:\Users\joven\miniconda3\lib\site-packages\transformers\dynamic_module_utils.py", line 157, in get_class_in_module shutil.copy(f"{module_dir}/{module_file_name}", tmp_dir) File "C:\Users\joven\miniconda3\lib\shutil.py", line 417, in copy copyfile(src, dst, follow_symlinks=follow_symlinks) File "C:\Users\joven\miniconda3\lib\shutil.py", line 254, in copyfile with open(src, 'rb') as fsrc: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\joven\\.cache\\huggingface\\modules\\transformers_modules\\chatglm2-6b/chatglm2-6b/tokenization_chatglm.py'

反正正反斜杠，全路径，都试过了。求好心人解答。

Expected Behavior

No response

Steps To Reproduce

根据文档步骤来的。 chatglm2-6b模型大小：23.2 GB

Environment

- OS:Windows11
- Python:py 3.10.10
- Transformers:4.27.1
- PyTorch:2.0
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

[Nico0416]

chatglm2-6b这个目录下是 https://huggingface.co/THUDM/chatglm2-6b/tree/main 这个里面的文件了吗

[mainxx]

@Nico0416 是的，一模一样，如图1：

但是我chatglm2-6b跟代码同一级。如图2：

图1：

图2：

[linzeyang]

I prefer to use the absolute path of the cloned model repository in web_demo.py so that I can run web_demo.py wherever I want. In your case you can try passing C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main\chatglm2-6b\ (or replace \ with \\? I'm not sure how exactly the path should be on Windows) to AutoTokenizer.from_pretrained() method.

hope this can help you.

[xuzhiang]

This is my code, and it can run on my Windows platform: tokenizer = AutoTokenizer.from_pretrained("D:\model\chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("D:\model\chatglm2-6b", trust_remote_code=True, device='cuda')

[mainxx]

@xuzhiang

Error: Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Traceback (most recent call last): File "C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main\web_demo.py", line 6, in model = AutoModel.from_pretrained("C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main\chatglm2-6b\", trust_remote_code=True, device='cuda') File "C:\Users\joven\miniconda3\lib\site-packages\transformers\models\auto\auto_factory.py", line 466, in from_pretrained return model_class.from_pretrained( File "C:\Users\joven\miniconda3\lib\site-packages\transformers\modeling_utils.py", line 2498, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\modeling_chatglm.py", line 767, in init self.transformer = ChatGLMModel(config, empty_init=empty_init, device=device) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\modeling_chatglm.py", line 690, in init self.embedding = init_method(Embedding, config, **init_kwargs) File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\utils\init.py", line 52, in skip_init return module_cls(*args, **kwargs).to_empty(device=final_device) File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1024, in to_empty return self._apply(lambda t: torch.empty_like(t, device=device)) File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply module._apply(fn) File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 820, in _apply param_applied = fn(param) File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1024, in return self.apply(lambda t: torch.empty_like(t, device=device)) File "C:\Users\joven\miniconda3\lib\site-packages\torch\cuda_init.py", line 239, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

是我显卡不支持的问题？4070Ti

[mainxx]

tokenizer = AutoTokenizer.from_pretrained("F:\model\chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("F:\model\chatglm2-6b", trust_remote_code=True, device='cuda')

This is the same as the mistake above

[ysyx2008]

你这是安装了CPU版本的torch吧，去官网装个cuda版本的。

[ysyx2008]

2我还没试，但1代这样配置是没有问题的，顺利使用。可以参考下。 win10系统，使用了相对路径。

[ysyx2008]

可能就是最后不要加斜杠或是反斜杠？

[ysyx2008]

另外，模型没有这么大吧，也就12G左右。

[mainxx]

@ysyx2008 好的。我试一试。

[dogvane]

感觉是 transformers 的问题，无法本地加载，可以试试下面的代码。

from pathlib import Path modelFile = 'G:\\GPT\\ChatGLM2-6B\\cache\\chatglm2-6b-int4' mf = Path(modelFile)

tokenizer = AutoTokenizer.from_pretrained(mf, trust_remote_code=True) model = AutoModel.from_pretrained(mf, trust_remote_code=True).cuda()

相对路径也能用 modelFile = './cache/chatglm2-6b-int4'

[mainxx]

@dogvane

解决了那个。我是这样解决的： conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia 之前用pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118，好像不行。 但是又遇到新问题： Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Traceback (most recent call last): File "C:\Users\joven\miniconda3\lib\site-packages\gradio\routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "C:\Users\joven\miniconda3\lib\site-packages\gradio\blocks.py", line 1352, in process_api result = await self.call_function( File "C:\Users\joven\miniconda3\lib\site-packages\gradio\blocks.py", line 1093, in call_function prediction = await utils.async_iteration(iterator) File "C:\Users\joven\miniconda3\lib\site-packages\gradio\utils.py", line 341, in async_iteration return await iterator.anext() File "C:\Users\joven\miniconda3\lib\site-packages\gradio\utils.py", line 334, in anext return await anyio.to_thread.run_sync( File "C:\Users\joven\miniconda3\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\joven\miniconda3\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\Users\joven\miniconda3\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, *args) File "C:\Users\joven\miniconda3\lib\site-packages\gradio\utils.py", line 317, in run_sync_iterator_async return next(iterator) File "C:\Users\joven\source\Github\ChatGLM2-6B-main\ChatGLM2-6B-main\web_demo.py", line 64, in predict for response, history, past_key_values in model.stream_chat(tokenizer, input, history, past_key_values=past_key_values, File "C:\Users\joven\miniconda3\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context response = gen.send(None) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 973, in stream_chat for outputs in self.stream_generate(**inputs, past_key_values=past_key_values, File "C:\Users\joven\miniconda3\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context response = gen.send(None) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 1057, in stream_generate outputs = self( File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 845, in forward transformer_outputs = self.transformer( File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 741, in forward hidden_states, presents, all_hidden_states, all_self_attentions = self.encoder( File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 588, in forward hidden_states, kv_cache = layer( File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 510, in forward attention_output, kv_cache = self.self_attention( File "C:\Users\joven\miniconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Users\joven/.cache\huggingface\modules\transformers_modules\chatglm2-6b\modeling_chatglm.py", line 374, in forward query_layer = apply_rotary_pos_emb(query_layer, rotary_pos_emb) RuntimeError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript (most recent call last): RuntimeError: nvrtc: error: failed to open nvrtc-builtins64_118.dll. Make sure that nvrtc-builtins64_118.dll is installed correctly.

[dogvane]

要不降一个版本到117再试试

[msly]

1 本地绝对路径要使用 \\ 不能用 / , 否则报错OSError: [WinError 123] 2 torch要用gpu版本的, pip uninstall torch pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 3 显存<9G 的下载int4版本 git clone https://huggingface.co/THUDM/chatglm2-6b-int4 10G内存可以用int8量化 model = AutoModel.from_pretrained("Z:\\chatglm2-6b", trust_remote_code=True).quantize(8).cuda()

[CyanMystery]

绝对路径相对路径 \，\ ,Path()， 都试过了 不行啊 Win10 cu118 tramsformers 4.27.1 chatglm6b就不会报错

[CyanMystery]

TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

[wupj1993]

绝对路径相对路径 \，\ ,Path()， 都试过了 不行啊 Win10 cu118 tramsformers 4.27.1 chatglm6b就不会报错

你模型（ https://huggingface.co/THUDM/chatglm2-6b/tree/main ）下面的文件是不是不完整，我之前也是不完整才导致一样的错，

[CyanMystery]

绝对路径相对路径 \，\ ,Path()， 都试过了 不行啊 Win10 cu118 tramsformers 4.27.1 chatglm6b就不会报错

你模型（ https://huggingface.co/THUDM/chatglm2-6b/tree/main ）下面的文件是不是不完整，我之前也是不完整才导致一样的错，

牛逼 确实文件少了 git lfs不是很懂用 之前都是手动下载的文件 多谢多谢

[mainxx]

git lfs下载到一半，能不能继续下载。经常都是下载到一半。

[CyanMystery]

git lfs下载到一半，能不能继续下载。经常都是下载到一半。

是这样的而且不显示进度 我都是直接用浏览器下载了

[baiyutang]

我用 wget

[moviewang]

tokenizer = AutoTokenizer.from_pretrained("D:\model\chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("D:\model\chatglm2-6b", trust_remote_code=True, device='cuda')

我的出现这个原因是少下载了tokenizer.model