报错You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []
PaddleNLP 2.3.3 PaddlePaddle-gpu 2.3.0.post112 win10 python 3.7.13 报错代码:result = ie(textlist) 补充代码:ie = Taskflow('information_extraction', schema=schema,task_path=./checkpoint/model_best) textlist是一个不含空串的字符串列表。 详细情况:ie是用uie_medium增量训练的模型,持续从数据库中拿出文本处理成字符串列表后传入ie,一开始没有问题,运行了一段时间后,在某份数据之后报了You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []这个错误。但是编写脚本单独测试报错报错数据的前后十份数据都没有问题,运行成功没有报错。查看其他issue发现这个报错的原因是数据脏,但我的数据都是正常的,没有乱七八糟的符号。因此想请问一下有哪些情况会导致这个报错?
@WethatVivy 升级paddlenlp到最新版本再试下
File "train/uie/utils.py", line 163, in convert_example encoded_inputs = tokenizer(text=[example["prompt"]], File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2233, in call return self.batch_encode( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2439, in batch_encode return self._batch_encode_plus( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils.py", line 1128, in _batch_encode_plus batch_outputs = self._batch_prepare_for_model( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils.py", line 1300, in _batch_prepare_for_model batch_outputs = self.pad( File "/root/anaconda3/lib/python3.9/site-packages/paddlenlp/transformers/tokenizer_utils_base.py", line 2566, in pad raise ValueError( ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided []
已经是最新版了,也报这个错
This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。
This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。