PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

报错You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []

Open WethatVivy opened this issue 3 years ago • 3 comments

PaddleNLP 2.3.3 PaddlePaddle-gpu 2.3.0.post112 win10 python 3.7.13 报错代码:result = ie(textlist) 补充代码:ie = Taskflow('information_extraction', schema=schema,task_path=./checkpoint/model_best) textlist是一个不含空串的字符串列表。 详细情况:ie是用uie_medium增量训练的模型,持续从数据库中拿出文本处理成字符串列表后传入ie,一开始没有问题,运行了一段时间后,在某份数据之后报了You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []这个错误。但是编写脚本单独测试报错报错数据的前后十份数据都没有问题,运行成功没有报错。查看其他issue发现这个报错的原因是数据脏,但我的数据都是正常的,没有乱七八糟的符号。因此想请问一下有哪些情况会导致这个报错?

WethatVivy avatar Jul 12 '22 01:07 WethatVivy

@WethatVivy 升级paddlenlp到最新版本再试下

linjieccc avatar Jul 12 '22 02:07 linjieccc

File "train/uie/utils.py", line 163, in convert_example encoded_inputs = tokenizer(text=[example["prompt"]], File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2233, in call return self.batch_encode( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2439, in batch_encode return self._batch_encode_plus( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils.py", line 1128, in _batch_encode_plus batch_outputs = self._batch_prepare_for_model( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils.py", line 1300, in _batch_prepare_for_model batch_outputs = self.pad( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2566, in pad raise ValueError( ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []

junphine avatar Nov 08 '22 05:11 junphine

已经是最新版了,也报这个错

junphine avatar Nov 08 '22 05:11 junphine

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

github-actions[bot] avatar Jan 08 '23 00:01 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。

github-actions[bot] avatar Jan 22 '23 00:01 github-actions[bot]