re模型预测报错

[icutool]

问题描述 / Problem Description

对表格图片标注训练ser和re之后，用re进行预测报错 对re模型进行模型导出后预测又会出现如下的报错

(paddlecpu) [root@hlwsjhjpt-018 /data/table/PaddleOCR/ppstructure]#  python /data/table/PaddleOCR/ppstructure/kie/predict_kie_token_ser_re.py --kie_algorithm=LayoutXLM --re_model_dir=../inference/re_vi_layoutxlm --ser_model_dir=../inference/ser_vi_layoutxlm --use_visual_backbone=False --image_dir=../test.png --ser_dict_path=../train_data/XFUND/class_list_xfun.txt --vis_font_path=../doc/fonts/simfang.ttf --ocr_order_method="tb-yx"
[2024-06-19 17:51:57,971] [    INFO] - tokenizer config file saved in /root/.paddlenlp/models/layoutxlm-base-uncased/tokenizer_config.json
[2024-06-19 17:51:57,971] [    INFO] - Special tokens file saved in /root/.paddlenlp/models/layoutxlm-base-uncased/special_tokens_map.json
E0619 17:51:57.993790 2210525 analysis_config.cc:125] Please use PaddlePaddle with GPU version.
E0619 17:51:59.371573 2210525 analysis_config.cc:125] Please use PaddlePaddle with GPU version.
Traceback (most recent call last):
  File "/data/table/PaddleOCR/ppstructure/kie/predict_kie_token_ser_re.py", line 144, in <module>
    main(parse_args())
  File "/data/table/PaddleOCR/ppstructure/kie/predict_kie_token_ser_re.py", line 105, in main
    re_res, elapse = ser_re_predictor(img)
  File "/data/table/PaddleOCR/ppstructure/kie/predict_kie_token_ser_re.py", line 68, in __call__
    self.predictor.run()
ValueError: In user code:

    File "/data/table/PaddleOCR/tools/export_model.py", line 295, in <module>
      main()
    File "/data/table/PaddleOCR/tools/export_model.py", line 289, in main
      export_single_model(
    File "/data/table/PaddleOCR/tools/export_model.py", line 192, in export_single_model
      paddle.jit.save(model, save_path)
    File "/data/conda/envs/251/lib/python3.9/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/api.py", line 752, in wrapper
      func(layer, path, input_spec, **configs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/dygraph/base.py", line 75, in __impl__
      return func(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/api.py", line 1043, in save
      static_func.concrete_program_specify_input_spec(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/program_translator.py", line 709, in concrete_program_specify_input_spec
      concrete_program, _ = self.get_concrete_program(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/program_translator.py", line 589, in get_concrete_program
      concrete_program, partial_program_layer = self._program_cache[
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/program_translator.py", line 1249, in __getitem__
      self._caches[item_id] = self._build_once(item)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/program_translator.py", line 1193, in _build_once
      concrete_program = ConcreteProgram.from_func_spec(
    File "/data/conda/envs/251/lib/python3.9/site-packages/decorator.py", line 232, in fun
      return caller(func, *(extras + args), **kw)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/dygraph/base.py", line 75, in __impl__
      return func(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/program_translator.py", line 1063, in from_func_spec
      outputs = static_func(*inputs)
    File "/data/table/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 84, in forward
      if self.use_backbone:
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 352, in convert_ifelse
      out = _run_py_ifelse(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 429, in _run_py_ifelse
      py_outs = true_fn() if pred else false_fn()
    File "/data/table/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 85, in forward
      x = self.backbone(x)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1256, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/data/table/PaddleOCR/ppocr/modeling/backbones/vqa_layoutlm.py", line 248, in forward
      x = self.model(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1256, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1412, in forward
      loss, pred_relations = self.extractor(sequence_output, entities, relations)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1256, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/nn/layer/layers.py", line 1235, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1309, in forward
      for b in range(batch_size):
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 117, in convert_while_loop
      _run_paddle_while(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 171, in _run_paddle_while
      loop_vars = control_flow.while_loop(new_cond_fn, new_body_fn, loop_vars)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/layers/control_flow.py", line 1085, in while_loop
      output_vars = body(*loop_vars)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 153, in new_body_fn
      body()
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddlenlp/transformers/layoutxlm/modeling.py", line 1339, in forward
      if len(pred_relations) > 0:
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 342, in convert_ifelse
      out = _run_paddle_cond(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/jit/dy2static/convert_operators.py", line 399, in _run_paddle_cond
      cond_outs = paddle.static.nn.cond(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/static/nn/control_flow.py", line 1133, in cond
      merged_output = list(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/static/nn/control_flow.py", line 1131, in merge_every_var_list
      return map_structure(partial(merge_func, name), false_vars, true_vars)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/utils/layers_utils.py", line 264, in map_structure
      return pack_sequence_as(structure[0], [func(*x) for x in entries])
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/utils/layers_utils.py", line 264, in <listcomp>
      return pack_sequence_as(structure[0], [func(*x) for x in entries])
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/static/nn/control_flow.py", line 1125, in <lambda>
      lambda name, false_var, true_var: select_input_with_buildin_type(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/static/nn/control_flow.py", line 1182, in select_input_with_buildin_type
      return select_input(inputs, mask)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/layers/control_flow.py", line 135, in select_input
      helper.append_op(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/layer_helper.py", line 45, in append_op
      return self.main_program.current_block().append_op(*args, **kwargs)
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/framework.py", line 4013, in append_op
      op = Operator(
    File "/data/conda/envs/251/lib/python3.9/site-packages/paddle/fluid/framework.py", line 2781, in __init__
      for frame in traceback.extract_stack():

    InvalidArgumentError: The Variable type must be N3phi11TensorArrayE, but the type it holds is N3phi11DenseTensorE.
      [Hint: Expected holder_->Type() == VarTypeTrait<T>::kId, but received holder_->Type():7 != VarTypeTrait<T>::kId:13.] (at /paddle/paddle/fluid/framework/variable.h:58)
      [operator < select_input > error]

运行环境 / Runtime Environment

• OS: ubuntu18.04
• Paddle: 2.5.1和2.6.1都试了
• PaddleOCR: 2.6 和最新的都试了

复现代码 / Reproduction Code

python ./tools/infer_kie_token_ser_re.py \
  -c configs/kie/vi_layoutxlm/re_vi_layoutxlm_xfund_zh.yml \
  -o Architecture.Backbone.checkpoints=./output/re_vi_layoutxlm_udml_xfund_zh/best_accuracy/ \
  Global.infer_img=./test.png \
  -c_ser configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yml \
  -o_ser Architecture.Backbone.checkpoints=output/ \
  ser_vi_layoutxlm_udml_xfund_zh/best_accuracy/