AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

Oct 10 '23 08:10 Hopping-Rabbit

同样问题 python3.9 transformers==4.34.0

Oct 10 '23 11:10 dongzeguang

我换成 transformers==4.33.3 没问题了，但报下面的错误了


欢迎使用百川大模型，输入进行对话，vim 多行输入，clear 清空历史，CTRL+C 中断生成，stream 开关流式生成，exit 结束。

用户：你好啊

Baichuan 2：Exception in thread Thread-3 (generate):
Traceback (most recent call last):
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/transformers/generation/utils.py", line 1648, in generate
    return self.sample(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/transformers/generation/utils.py", line 2730, in sample
    outputs = self(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 693, in forward
    outputs = self.model(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 467, in forward
    layer_outputs = decoder_layer(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 244, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 147, in forward
    proj = self.W_pack(hidden_states)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'


exit

Oct 11 '23 01:10 chenlingmin

碰到了同样的问题，按照官方提供的运行了 pip install -r requirements.txt的代码，在运行web_demo和cil_demo时包括推理代码时都碰到了AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'的报错，然后根据上面的朋友说的将transformers变为4.33.3时推理代码可以运行，但是web_demo和cil_demo在运行时可以加载，但是你跟他说一句话之后也会弹出RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'的问题，应该是有一些包的版本会冲突导致的问题，希望官方能尽快给出解决答案。

Oct 11 '23 03:10 JACKzhuz

碰到了同样的问题，按照官方提供的运行了 pip install -r requirements.txt的代码，在运行web_demo和cil_demo时包括推理代码时都碰到了AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'的报错，然后根据上面的朋友说的将transformers变为4.33.3时推理代码可以运行，但是web_demo和cil_demo在运行时可以加载，但是你跟他说一句话之后也会弹出RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'的问题，应该是有一些包的版本会冲突导致的问题，希望官方能尽快给出解决答案。

transformers变为4.33.3，同时把torch和triton版本改为2.0的, 我这边可以解决上面的报错

Oct 11 '23 05:10 Janady

碰到了同样的问题，按照官方提供的运行了 pip install -r requirements.txt的代码，在运行web_demo和cil_demo时包括推理代码时都碰到了AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'的报错，然后根据上面的朋友说的将transformers变为4.33.3时推理代码可以运行，但是web_demo和cil_demo在运行时可以加载，但是你跟他说一句话之后也会弹出RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'的问题，应该是有一些包的版本会冲突导致的问题，希望官方能尽快给出解决答案。

transformers变为4.33.3，同时把torch和triton版本改为2.0的, 我这边可以解决上面的报错

根据你的建议，我按照你的步骤去尝试了一下，torch=2.0.1，trtton=2.0.0，transformers=4.33.3，可以运行cli_demo.py，但是在你打一个招呼之后，还是会出现报错RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

Oct 11 '23 05:10 JACKzhuz

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

Oct 11 '23 06:10 Xu-Chen

我换成 transformers==4.33.3 没问题了，但报下面的错误了


欢迎使用百川大模型，输入进行对话，vim 多行输入，clear 清空历史，CTRL+C 中断生成，stream 开关流式生成，exit 结束。

用户：你好啊

Baichuan 2：Exception in thread Thread-3 (generate):
Traceback (most recent call last):
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/transformers/generation/utils.py", line 1648, in generate
    return self.sample(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/transformers/generation/utils.py", line 2730, in sample
    outputs = self(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 693, in forward
    outputs = self.model(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 467, in forward
    layer_outputs = decoder_layer(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 244, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 147, in forward
    proj = self.W_pack(hidden_states)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/xxxxxx/anaconda3/envs/baichuan/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'


exit

你的问题解决了吗兄弟？

Oct 11 '23 06:10 JACKzhuz

在微调时遇到了同样的问题

Oct 11 '23 08:10 cuisws

这里也有一样的问题，采用的是把 super() 修改到最后执行这个方法成功了，没有改变transformers的安装版本，torch=2.0.1，trtton=2.0.0

Oct 12 '23 03:10 ruiqianheartseed

解决方案

transformers变为4.33.3，同时把torch和triton版本改为2.0的, 我这边可以解决上面的报错

pip install transformers==4.33.3 pip install torch=2.0.0

Oct 12 '23 04:10 KissMyLady

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

这个方案可行，不需要改环境。官方应该修订一下，更新版本吧。

Oct 16 '23 15:10 metercai

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

这个方案可行，不需要改环境。官方应该修订一下，更新版本吧。

修改后，执行，还是会报错。执行非量化的，不会报错。但是执行4bits的会报这个错。

Oct 23 '23 03:10 huangdang111

pip install transformers==4.33.1 tokenizers==0.13.3

Nov 10 '23 10:11 li-haojun

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

4.35版本transformers ，这样修改之后，加载模型还会遇到另外一个问题： TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'

Nov 13 '23 07:11 linxxx3

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

Nov 15 '23 07:11 chenkang404

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

如果你不指定本机目录，AutoTokenizer会从默认缓存地址找代码和模型参数加载，默认缓存是HF_MODULES_CACHE='~/.cache/huggingface/modules'

Nov 15 '23 07:11 nlp4whp

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

如果你不指定本机目录，AutoTokenizer会从默认缓存地址找代码和模型参数加载，默认缓存是HF_MODULES_CACHE='~/.cache/huggingface/modules'

“指定本机目录”是什么意思，我用vllm加载的

Nov 15 '23 07:11 chenkang404

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

如果你不指定本机目录，AutoTokenizer会从默认缓存地址找代码和模型参数加载，默认缓存是HF_MODULES_CACHE='~/.cache/huggingface/modules'

“指定本机目录”是什么意思，我用vllm加载的

就是传给vllm的--tokenizer参数，如果你传的是baichuan-inc/Baichuan2-13B-Chat，他就会帮你下载到~/.cache/huggingface/modules

Nov 15 '23 07:11 nlp4whp

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

如果你不指定本机目录，AutoTokenizer会从默认缓存地址找代码和模型参数加载，默认缓存是HF_MODULES_CACHE='~/.cache/huggingface/modules'

“指定本机目录”是什么意思，我用vllm加载的

就是传给vllm的--tokenizer参数，如果你传的是baichuan-inc/Baichuan2-13B-Chat，他就会帮你下载到~/.cache/huggingface/modules

传的是一个本地路径，但这个本地路径是一个绝对路径“./root/../Baichuan2-13B-Chat”，但先前用transformers=4.34.0.dev可以加载，刚才我更新了版本到4.35.1，报上述错，换回4.34.0还报上述错，是不是需要dev版本，忘了怎么安装dev版本了

Nov 15 '23 07:11 chenkang404

ile "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model' 为什么会加载".cache"中的transformers模块

如果你不指定本机目录，AutoTokenizer会从默认缓存地址找代码和模型参数加载，默认缓存是HF_MODULES_CACHE='~/.cache/huggingface/modules'

“指定本机目录”是什么意思，我用vllm加载的

就是传给vllm的--tokenizer参数，如果你传的是baichuan-inc/Baichuan2-13B-Chat，他就会帮你下载到~/.cache/huggingface/modules

传的是一个本地路径，但这个本地路径是一个绝对路径“./root/../Baichuan2-13B-Chat”，但先前用transformers=4.34.0.dev可以加载，刚才我更新了版本到4.35.1，报上述错，换回4.34.0还报上述错，是不是需要dev版本，忘了怎么安装dev版本了

好吧，是transformer和vllm版本不匹配问题

Nov 15 '23 09:11 chenkang404

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

4.35版本transformers ，这样修改之后，加载模型还会遇到另外一个问题： TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'

我也遇到了，你解决了吗

Nov 16 '23 03:11 Leawnn

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

        self.vocab_file = vocab_file
        self.add_bos_token = add_bos_token
        self.add_eos_token = add_eos_token
        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        self.sp_model.Load(vocab_file)
        super().__init__(
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            pad_token=pad_token,
            add_bos_token=add_bos_token,
            add_eos_token=add_eos_token,
            sp_model_kwargs=self.sp_model_kwargs,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
            **kwargs,
        )
        # self.vocab_file = vocab_file
        # self.add_bos_token = add_bos_token
        # self.add_eos_token = add_eos_token
        # self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
        # self.sp_model.Load(vocab_file)

4.35版本transformers ，这样修改之后，加载模型还会遇到另外一个问题： TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'

transformers版本为4.34.0，就可以了

Nov 16 '23 03:11 Leawnn

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

这个方法可行，墙裂推荐，其他人的方法都不太对

Nov 16 '23 03:11 lianglinyi

把transformers的版本换到4.33.1就行了。

Nov 21 '23 06:11 Eternal-Yan

使用的是web_demo.py启动，在碰到

RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

错误时，替换transformers=4.33.3,torch=2.0.0后不生效，依然还是这个错误。

翻看文档，把web_demo.py文件中

@st.cache_resource
def init_model():
    model = AutoModelForCausalLM.from_pretrained(
        "baichuan-inc/Baichuan2-13B-Chat",
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True,
        offload_folder="offload",
        offload_state_dict=True,
    )

torch_dtype=torch.float16,修改成torch_dtype=torch.bfloat16,后使用正常

Nov 25 '23 01:11 litterGuy

为什么我修改tokenization_baichuan.py之后，重新运行就把super（）又自动改到上面去了？

Nov 30 '23 11:11 tommy3266

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

这个方法可行，墙裂推荐，其他人的方法都不太对

大佬，为什么我修改tokenization_baichuan.py之后，重新运行就把super（）又自动改到上面去了？

Nov 30 '23 12:11 tommy3266

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

这个方法可行，墙裂推荐，其他人的方法都不太对

大佬，为什么我修改tokenization_baichuan.py之后，重新运行就把super（）又自动改到上面去了？

是加载已经下载好的模型吧？不会是重新下载模型？

Nov 30 '23 13:11 lianglinyi

修改下 tokenization_baichuan.py 也可以，把 super() 修改到最后执行

这个方法可行，墙裂推荐，其他人的方法都不太对

大佬，为什么我修改tokenization_baichuan.py之后，重新运行就把super（）又自动改到上面去了？

是加载已经下载好的模型吧？不会是重新下载模型？

是的，是加载本地模型的，

Dec 01 '23 00:12 tommy3266

量化版本还无法加载：

File ~/.cache\huggingface\modules\transformers_modules\baichuan-inc\Baichuan2-13B-Chat-4bits\961384bb9380d164c41d069b9012c6e1525b4f27\modeling_baichuan.py:611, in BaichuanForCausalLM.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs) 609 from accelerate.utils import get_balanced_memory 610 except ImportError: --> 611 raise ImportError(f"Needs import model weight init func to run quantize.") 612 # Instantiate model. 613 init_contexts = [no_init_weights(_enable=True)]

ImportError: Needs import model weight init func to run quantize.

Dec 03 '23 23:12 metero20000

Baichuan2 Baichuan2 copied to clipboard

AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

解决方案

Baichuan2
Baichuan2 copied to clipboard