MiniCPM
MiniCPM copied to clipboard
OpenBMB
[Bad Case]: 使用minicpm4-0.5B和8B的模型都报错
Description / 描述
你好，请问我在将minicpm4-0.5B和minicpm4-8B的模型下载下来之后，使用范例代码推理：
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)
path = "/mnt/2/haochen/LLM/MiniCPM/pretrained_models/MiniCPM4-0.5B"
device = "cuda"
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)
responds, history = model.chat(tokenizer, "Write an article about Artificial Intelligence.", temperature=0.7, top_p=0.7)
print(responds)
均报错如下：
(llm_minicpm4) u@u:/mnt/2/haochen/LLM/MiniCPM$ python transformers_inference_small.py Traceback (most recent call last): File "/mnt/2/haochen/LLM/MiniCPM/transformers_inference_small.py", line 7, in <module> tokenizer = AutoTokenizer.from_pretrained(path) File "/mnt/2/haochen/envs/llm_minicpm4/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 880, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) File "/mnt/2/haochen/envs/llm_minicpm4/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2110, in from_pretrained return cls._from_pretrained( File "/mnt/2/haochen/envs/llm_minicpm4/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2336, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/mnt/2/haochen/envs/llm_minicpm4/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 159, in __init__ super().__init__( File "/mnt/2/haochen/envs/llm_minicpm4/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 114, in __init__ fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file) Exception: data did not match any variant of untagged enum ModelWrapper at line 490842 column 3
但是我用MiniCPM-2B-sft-bf16就可以正常inference，请问该如何解决这个问题呢？谢谢！
Case Explaination / 案例解释
No response
这可能是由于tokenizers和transformers库的版本不一致导致的。你可以尝试更新tokenizers库。如果您使用的是MiniCPM4进行推理，可以尝试将tokenizers更新至版本0.20.0。如果问题仍然存在，请把相关库的信息提供给我。
