JittorLLMs
JittorLLMs copied to clipboard
Jittor
运行报错
运行信息如下
(lvm) root@debian:/opt/JittorLLMs# python cli_demo.py chatglm
[i 0505 14:20:32.927485 48 compiler.py:955] Jittor(1.3.7.15) src: /root/anaconda3/envs/lvm/lib/python3.8/site-packages/jittor
[i 0505 14:20:32.930535 48 compiler.py:956] g++ at /usr/bin/g++(10.2.1)
[i 0505 14:20:32.930623 48 compiler.py:957] cache_path: /root/.cache/jittor/jt1.3.7/g++10.2.1/py3.8.16/Linux-5.10.0-2xd8/IntelRCoreTMi7x5d/default
[i 0505 14:20:32.940381 48 __init__.py:411] Found addr2line(2.35.2) at /usr/bin/addr2line.
[i 0505 14:20:33.164447 48 __init__.py:227] Total mem: 5.76GB, using 1 procs for compiling.
[i 0505 14:20:33.427746 48 jit_compiler.cc:28] Load cc_path: /usr/bin/g++
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [01:19<00:00, 9.90s/it]Traceback (most recent call last):
File "cli_demo.py", line 8, in <module>
model = models.get_model(args)
File "/opt/JittorLLMs/models/__init__.py", line 46, in get_model
return module.get_model(args)
File "/opt/JittorLLMs/models/chatglm/__init__.py", line 48, in get_model
return ChatGLMMdoel(args)
File "/opt/JittorLLMs/models/chatglm/__init__.py", line 26, in __init__
self.model.float32()
File "/root/anaconda3/envs/lvm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'ChatGLMForConditionalGeneration' object has no attribute 'float32'
我这里直接0%挂掉，centos 7， 2H2G Loading checkpoint shards: 0%| | 0/8 [00:58<?, ?it/s] Traceback (most recent call last): File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict return torch.load(checkpoint_file, map_location="cpu") File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 809, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1172, in _load result = unpickler.load() File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1142, in persistent_load typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location)) File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1112, in load_tensor storage = zip_file.get_storage_from_record(name, numel, torch.UntypedStorage)._typed_storage()._untyped_storage RuntimeError: [enforce fail at alloc_cpu.cpp:75] err == 0. DefaultCPUAllocator: can't allocate memory: you tried to allocate 100663296 bytes. Error code 12 (Cannot allocate memory)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 419, in load_state_dict if f.read(7) == "version": File "/usr/local/python3.10/lib/python3.10/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/JittorLLMs/cli_demo.py", line 8, in
一样报float32
Loading checkpoint shards: 100%|██████████████████████████████████████| 8/8 [00:43<00:00, 5.43s/it]
Traceback (most recent call last):
File "C:\JittorLLMs-main\cli_demo.py", line 8, in
chatglm/init.py中，self.model.float32()改为self.model.float()，试一下。
感谢大佬指点，修改后确实可以跑了！，期间还遇到空格TAB混用和内存不足两个问题，那应该就是我自己的问题了😂
我这里直接0%挂掉，centos 7， 2H2G Loading checkpoint shards: 0%| | 0/8 [00:58<?, ?it/s] Traceback (most recent call last): File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict return torch.load(checkpoint_file, map_location="cpu") File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 809, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1172, in _load result = unpickler.load() File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1142, in persistent_load typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location)) File "/usr/local/python3.10/lib/python3.10/site-packages/torch/serialization.py", line 1112, in load_tensor storage = zip_file.get_storage_from_record(name, numel, torch.UntypedStorage)._typed_storage()._untyped_storage RuntimeError: [enforce fail at alloc_cpu.cpp:75] err == 0. DefaultCPUAllocator: can't allocate memory: you tried to allocate 100663296 bytes. Error code 12 (Cannot allocate memory)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 419, in load_state_dict if f.read(7) == "version": File "/usr/local/python3.10/lib/python3.10/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/root/JittorLLMs/cli_demo.py", line 8, in model = models.get_model(args) File "/root/JittorLLMs/models/init.py", line 46, in get_model return module.get_model(args) File "/root/JittorLLMs/models/chatglm/init.py", line 48, in get_model return ChatGLMMdoel(args) File "/root/JittorLLMs/models/chatglm/init.py", line 22, in init self.model = AutoModel.from_pretrained(os.path.dirname(file), trust_remote_code=True) File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 459, in from_pretrained return model_class.from_pretrained( File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2478, in from_pretrained ) = cls._load_pretrained_model( File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2780, in _load_pretrained_model state_dict = load_state_dict(shard_file) File "/usr/local/python3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 431, in load_state_dict raise OSError( OSError: Unable to load weights from pytorch checkpoint file for '/root/JittorLLMs/models/chatglm/pytorch_model-00001-of-00008.bin' at '/root/JittorLLMs/models/chatglm/pytorch_model-00001-of-00008.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
可以更新下包：
pip install --upgrade torch torchvision
