uie_pytorch icon indicating copy to clipboard operation
uie_pytorch copied to clipboard

convert uie-m-base报错AttributeError: 'ErnieMTokenizer' object has no attribute 'vocab'

Open CastJo opened this issue 1 year ago • 2 comments

Traceback (most recent call last): File "/root/autodl-tmp/uie_pytorch/uie_predictor.py", line 679, in uie = UIEPredictor(model=args.model, task_path=args.task_path, schema_lang=args.schema_lang, schema=args.schema, engine=args.engine, device=args.device, File "/root/autodl-tmp/uie_pytorch/uie_predictor.py", line 147, in init self._prepare_predictor() File "/root/autodl-tmp/uie_pytorch/uie_predictor.py", line 162, in _prepare_predictor self._tokenizer = ErnieMTokenizerFast.from_pretrained( File "/root/autodl-tmp/conda/envs/uie_torch_cpu/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in from_pretrained return cls._from_pretrained( File "/root/autodl-tmp/conda/envs/uie_torch_cpu/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2049, in _from_pretrained slow_tokenizer = (cls.slow_tokenizer_class)._from_pretrained( File "/root/autodl-tmp/conda/envs/uie_torch_cpu/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2249, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/root/autodl-tmp/uie_pytorch/tokenizer.py", line 139, in init super().init( File "/root/autodl-tmp/conda/envs/uie_torch_cpu/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 367, in init self._add_tokens( File "/root/autodl-tmp/conda/envs/uie_torch_cpu/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 467, in _add_tokens current_vocab = self.get_vocab().copy() File "/root/autodl-tmp/uie_pytorch/tokenizer.py", line 185, in get_vocab return dict(self.vocab, **self.added_tokens_encoder) AttributeError: 'ErnieMTokenizer' object has no attribute 'vocab'

CastJo avatar Oct 24 '23 06:10 CastJo