bert模型的paddlelite推理

[railgun-zyy]

运行时报错： 1 [F 8/21 15: 8:20.763 ...-Lite/lite/operators/fill_constant_op.cc:44 InferShapeImpl] no valid out_shape. Must set one of shape_tensor, or shape_tensor_list, or shape

代码： from paddlelite.lite import * import numpy as np import cv2 config = MobileConfig()

config.set_model_from_file("bert-base-chinese.nb") predictor = create_paddle_predictor(config)

input_ids = predictor.get_input(0) token_type_ids = predictor.get_input(1) input = np.zeros([1,1],dtype=np.int64) token_type = np.zeros([1,1],dtype=np.int64) input_ids.from_numpy(input) token_type_ids.from_numpy(token_type) print(1) predictor.run() print(2) output_tensor = predictor.get_output(0).numpy() print(output_tensor.shape)

[engineer1109]

用C++版本，int64应该没处理方式

[DustinPT]

@railgun-zyy 我也遇到了这个问题，你解决了吗？

[hong19860320]

可能原因：新版 Paddle 产出的模型，fill_constant 算子在表示 shape 为 [1] 时 shape 属性变成了空，导致 shape 推导失败。 解决方案： 方案1： Paddle Lite 增加对 fill_constant 算子空 shape 属性的支持，就是修改这个文件 https://github.com/PaddlePaddle/Paddle-Lite/blob/3c61295edf66eb5c24e7c98b604c14a1bd4457d7/lite/operators/fill_constant_op.cc#L56 。 方案2：参考 issue https://github.com/PaddlePaddle/Paddle-Lite/issues/10438 我们提供的 modify_model.py.zip 脚本处理一下原始的 paddle 模型。

[engineer1109]

@hong19860320 你是不会改代码了吗？ 0dim Tensor的问题解决很简单.

bool FillConstantOp::InferShapeImpl() const {
  std::vector<int64_t> out_shape;
  auto shape_tensor = param_.shape_tensor;
  auto shape_tensor_list = param_.shape_tensor_list;
  if (shape_tensor != nullptr) {
    auto shape_tensor_data = shape_tensor->data<int>();
    for (int i = 0; i < shape_tensor->numel(); i++) {
      out_shape.push_back(shape_tensor_data[i]);
    }
  } else if (!shape_tensor_list.empty()) {
    for (size_t i = 0; i < shape_tensor_list.size(); i++) {
      out_shape.push_back(shape_tensor_list[i]->data<int>()[0]);
    }
  } else if (!param_.shape.empty()) {
    out_shape = param_.shape;
  } else if (param_.shape.empty()) {
    out_shape = {1};
  } else {
    LOG(WARNING) << "FillConstant is 0D-tensor output";
  }

  param_.out->Resize(out_shape);
  return true;
}

加个

  } else if (param_.shape.empty()) {
    out_shape = {1};

现在Paddle不支持Paddle-Lite是什么原因？

[hong19860320]

@hong19860320 你是不会改代码了吗？ 0dim Tensor的问题解决很简单.

bool FillConstantOp::InferShapeImpl() const {
  std::vector<int64_t> out_shape;
  auto shape_tensor = param_.shape_tensor;
  auto shape_tensor_list = param_.shape_tensor_list;
  if (shape_tensor != nullptr) {
    auto shape_tensor_data = shape_tensor->data<int>();
    for (int i = 0; i < shape_tensor->numel(); i++) {
      out_shape.push_back(shape_tensor_data[i]);
    }
  } else if (!shape_tensor_list.empty()) {
    for (size_t i = 0; i < shape_tensor_list.size(); i++) {
      out_shape.push_back(shape_tensor_list[i]->data<int>()[0]);
    }
  } else if (!param_.shape.empty()) {
    out_shape = param_.shape;
  } else if (param_.shape.empty()) {
    out_shape = {1};
  } else {
    LOG(WARNING) << "FillConstant is 0D-tensor output";
  }

  param_.out->Resize(out_shape);
  return true;
}

加个

  } else if (param_.shape.empty()) {
    out_shape = {1};

现在Paddle不支持Paddle-Lite是什么原因？

这个只是 workaround，可以解决部分问题，还要考虑各个 pass 对 0-dims 的支持。

[hong19860320]

开源项目欢迎大家贡献代码 @engineer1109