Pangu 2.6b 启动失败。

[Liufeiran123]

Traceback (most recent call last): File "/mnt/d/ai/RLHF/test.py", line 3, in tokenizer = AutoTokenizer.from_pretrained("/mnt/d/ai/pretrain_models/pangu", trust_remote_code=True) File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 774, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2028, in from_pretrained return cls._from_pretrained( File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2260, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/home/liu/.cache/huggingface/modules/transformers_modules/pangu/tokenization_gptpangu.py", line 28, in init super().init(**kwargs) File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 367, in init self._add_tokens( File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 467, in _add_tokens current_vocab = self.get_vocab().copy() File "/home/liu/anaconda3/envs/rlhf/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1675, in get_vocab raise NotImplementedError() NotImplementedError

[sunzeyeah]

需要从这里下载pangu-2.6B模型文件

[18780165950]

需要从这里下载pangu-2.6B模型文件

好像是vocab.model有问题，加载不了，我用的pangu-350M也是这个问题

[18780165950]

需要从这里下载pangu-2.6B模型文件

好像是vocab.model有问题，加载不了，我用的pangu-350M也是这个问题

版本问题，我降了了transformers版本，然后就ok了。