Baichuan-7B
Baichuan-7B copied to clipboard
baichuan-inc
有关哪个模型推理
Required prerequisites
- [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
- [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
- [X] Consider asking first in a Discussion.
Questions
你好，请问我用 https://huggingface.co/baichuan-inc/baichuan-7B/tree/main 上的模型推理，出现错误： Traceback (most recent call last): File "/home/ctgcdt/anaconda3/envs/baichuanGPT/lib/python3.9/site-packages/transformers/modeling_utils.py", line 446, in load_state_dict return torch.load(checkpoint_file, map_location="cpu") File "/home/ctgcdt/anaconda3/envs/baichuanGPT/lib/python3.9/site-packages/torch/serialization.py", line 797, in load with _open_zipfile_reader(opened_file) as opened_zipfile: File "/home/ctgcdt/anaconda3/envs/baichuanGPT/lib/python3.9/site-packages/torch/serialization.py", line 283, in init super().init(torch._C.PyTorchFileReader(name_or_buffer)) RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/ctgcdt/anaconda3/envs/baichuanGPT/lib/python3.9/site-packages/transformers/modeling_utils.py", line 450, in load_state_dict if f.read(7) == "version": File "/home/ctgcdt/anaconda3/envs/baichuanGPT/lib/python3.9/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ctgcdt/zhulei/NLP/baichuan-7B/test.py", line 5, in
请问这个模型是tf训练的吗，要在哪里修改呢
Checklist
- [X] I have provided all relevant and necessary information above.
- [X] I have chosen a suitable title for this issue.
推理代码： from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("baichuan-inc/baichuan-7B", device_map="auto", trust_remote_code=True) inputs = tokenizer('登鹳雀楼->王之涣\n夜雨寄北->', return_tensors='pt') inputs = inputs.to('cuda:0') pred = model.generate(**inputs, max_new_tokens=64,repetition_penalty=1.1) print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
