FastDeploy icon indicating copy to clipboard operation
FastDeploy copied to clipboard
verified publisher icon PaddlePaddle

运行编译出来的Csharp示例代码./FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/csharp/build/infer_demo.sln，出现推理报错闪退问题

Open 1wang11lijian1 opened this issue 1 year ago • 4 comments

温馨提示：根据社区不完全统计，按照模板提问，可以加快回复和解决问题的速度

环境

  • 【FastDeploy版本】： 自行编译FastDeploy SDK 的Csharp API接口
  • 【编译命令】
  • C++ SDK编译安装（Windows） cd E:\Work\OurDevelop\OCR\FastDeploy\build cmake .. -G "Visual Studio 16 2019" -A x64 -DENABLE_ORT_BACKEND=ON -DENABLE_PADDLE_BACKEND=ON -DENABLE_OPENVINO_BACKEND=ON -DENABLE_TRT_BACKEND=ON -DENABLE_VISION=ON -DENABLE_TEXT=ON -DWITH_GPU=ON -DWITH_CAPI=ON -DWITH_CSHARPAPI=ON -DTRT_DIRECTORY="E:\Work\Dependency_Library\TensorRT-8.6.1.6" -DCUDA_DIRECTORY="D:\SoftWare\CUDA\v11.3" -DCMAKE_INSTALL_PREFIX="E:\Work\OurDevelop\OCR\compiled_fastdeploy"
  • 编译Csharp示例代码 cd E:\Work\OurDevelop\OCR\FastDeploy\examples\vision\ocr\PP-OCR\cpu-gpu\csharp\build cmake .. -G "Visual Studio 16 2019" -A x64 -DFASTDEPLOY_INSTALL_DIR=E:\Work\OurDevelop\OCR\compiled_fastdeploy -DCUDA_DIRECTORY="D:/SoftWare/CUDA/v11.3" nuget restore msbuild infer_demo.sln /m:4 /p:Configuration=Release /p:Platform=x64
  • FastDeploy依赖的库拷贝至可执行程序所在目录 ./fastdeploy_init.bat install %cd% E:\Work\OurDevelop\OCR\FastDeploy\examples\vision\ocr\PP-OCR\cpu-gpu\csharp\build\Release
  • 【系统平台】: Windows x64(Windows10)
  • 【硬件】： Nvidia GPU 4060， CUDA 11.3 CUDNN 8.2
  • 【编译语言】：Csharp

问题日志及出现问题的操作流程

运行Csharp示例代码./FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/csharp/build/infer_demo.sln项目 Csharp运行代码如下

using System;
using System.IO;
using System.Runtime.InteropServices;
using OpenCvSharp;
using fastdeploy;

namespace Test
{
    public class TestPPOCRv3
    {
        public static void Main(string[] args)
        {
            string det_model_dir = "E:\\model_en\\en_PP-OCRv3_det_infer";
            string cls_model_dir = "E:\\model_en\\ch_ppocr_mobile_v2.0_cls_infer";
            string rec_model_dir = "E:\\model_en\\en_PP-OCRv3_rec_infer";
            string rec_label_file = "E:\\model_en\\en_dict.txt";
            string image_path = "E:\\model\\img_12.jpg";
            RuntimeOption runtimeoption = new RuntimeOption();
            int device_option = 1;
            if(device_option==0){
                runtimeoption.UseCpu();
            }else{
                runtimeoption.UseGpu();
            }
            string sep = "\\";
            string det_model_file = det_model_dir + sep + "inference.pdmodel";
            string det_params_file = det_model_dir + sep + "inference.pdiparams";

            string cls_model_file = cls_model_dir + sep + "inference.pdmodel";
            string cls_params_file = cls_model_dir + sep + "inference.pdiparams";

            string rec_model_file = rec_model_dir + sep + "inference.pdmodel";
            string rec_params_file = rec_model_dir + sep + "inference.pdiparams";

            fastdeploy.vision.ocr.DBDetector dbdetector = new fastdeploy.vision.ocr.DBDetector(det_model_file, det_params_file, runtimeoption, ModelFormat.PADDLE);
            fastdeploy.vision.ocr.Classifier classifier = new fastdeploy.vision.ocr.Classifier(cls_model_file, cls_params_file, runtimeoption, ModelFormat.PADDLE);
            fastdeploy.vision.ocr.Recognizer recognizer = new fastdeploy.vision.ocr.Recognizer(rec_model_file, rec_params_file, rec_label_file, runtimeoption, ModelFormat.PADDLE);
            fastdeploy.pipeline.PPOCRv3 model = new fastdeploy.pipeline.PPOCRv3(dbdetector, classifier, recognizer);
            if(!model.Initialized()){
                Console.WriteLine("Failed to initialize.\n");
            }
            Mat image = Cv2.ImRead(image_path);
            fastdeploy.vision.OCRResult res = model.Predict(image);
            Console.WriteLine(res.ToString());
            Mat res_img = fastdeploy.vision.Visualize.VisOcr(image, res);
            Cv2.ImShow("result.png", res_img);
            Cv2.ImWrite("result.png", res_img);
            Cv2.WaitKey(0);
        }
    }
}

在这行fastdeploy.vision.OCRResult res = model.Predict(image);出现报错 System.ArgumentOutOfRangeException:“在多字节的目标代码页中，没有此 Unicode 字符可以映射到的字符。 (异常来自 HRESULT:0x80070459)” 在我的电脑上是一直都会报这个问题，或者直接闪退./FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/csharp/build/Debug/infer_demo.exe (进程 31404)已退 出，代码为 -1073740940。 在我伙伴的电脑上有时图片正常计算（偶尔出现中文乱码问题），有时也报这个错误 如下图（偶尔出现中文乱码问题） 微信图片_20240705161958 如下图（计算报错闪退） 微信图片_20240705160728

所以想请问各位这是由于什么原因导致这两个问题，特在此求助大家！

1wang11lijian1 avatar Jul 05 '24 08:07 1wang11lijian1

这是capi中定义的OCRResult：

typedef struct FD_C_OCRResult {
  FD_C_TwoDimArrayInt32 boxes;
  FD_C_OneDimArrayCstr text;
  FD_C_OneDimArrayFloat rec_scores;
  FD_C_OneDimArrayFloat cls_scores;
  FD_C_OneDimArrayInt32 cls_labels;
  FD_C_TwoDimArrayInt32 table_boxes;
  FD_C_OneDimArrayCstr  table_structure;
  FD_C_Cstr table_html;
  FD_C_ResultType type;
} FD_C_OCRResult;

后面的C++的OCRResult到C的OCRResult有部分参数没有赋值

void FD_C_OCRResultWrapperToCResult(
    __fd_keep FD_C_OCRResultWrapper* fd_c_ocr_result_wrapper,
    __fd_keep FD_C_OCRResult* fd_c_ocr_result) {
  auto& ocr_result =
      CHECK_AND_CONVERT_FD_TYPE(OCRResultWrapper, fd_c_ocr_result_wrapper);
  // copy boxes
  const int boxes_coordinate_dim = 8;
  fd_c_ocr_result->boxes.size = ocr_result->boxes.size();
  fd_c_ocr_result->boxes.data =
      new FD_C_OneDimArrayInt32[fd_c_ocr_result->boxes.size];
  for (size_t i = 0; i < ocr_result->boxes.size(); i++) {
    fd_c_ocr_result->boxes.data[i].size = boxes_coordinate_dim;
    fd_c_ocr_result->boxes.data[i].data = new int[boxes_coordinate_dim];
    for (size_t j = 0; j < boxes_coordinate_dim; j++) {
      fd_c_ocr_result->boxes.data[i].data[j] = ocr_result->boxes[i][j];
    }
  }
  // copy text
  fd_c_ocr_result->text.size = ocr_result->text.size();
  fd_c_ocr_result->text.data = new FD_C_Cstr[fd_c_ocr_result->text.size];
  for (size_t i = 0; i < ocr_result->text.size(); i++) {
    fd_c_ocr_result->text.data[i].size = ocr_result->text[i].length();
    fd_c_ocr_result->text.data[i].data =
        new char[ocr_result->text[i].length() + 1];
    strncpy(fd_c_ocr_result->text.data[i].data, ocr_result->text[i].c_str(),
            ocr_result->text[i].length());
  }

  // copy rec_scores
  fd_c_ocr_result->rec_scores.size = ocr_result->rec_scores.size();
  fd_c_ocr_result->rec_scores.data =
      new float[fd_c_ocr_result->rec_scores.size];
  memcpy(fd_c_ocr_result->rec_scores.data, ocr_result->rec_scores.data(),
         sizeof(float) * fd_c_ocr_result->rec_scores.size);
  // copy cls_scores
  fd_c_ocr_result->cls_scores.size = ocr_result->cls_scores.size();
  fd_c_ocr_result->cls_scores.data =
      new float[fd_c_ocr_result->cls_scores.size];
  memcpy(fd_c_ocr_result->cls_scores.data, ocr_result->cls_scores.data(),
         sizeof(float) * fd_c_ocr_result->cls_scores.size);
  // copy cls_labels
  fd_c_ocr_result->cls_labels.size = ocr_result->cls_labels.size();
  fd_c_ocr_result->cls_labels.data =
      new int32_t[fd_c_ocr_result->cls_labels.size];
  memcpy(fd_c_ocr_result->cls_labels.data, ocr_result->cls_labels.data(),
         sizeof(int32_t) * fd_c_ocr_result->cls_labels.size);
  // copy type
  fd_c_ocr_result->type = static_cast<FD_C_ResultType>(ocr_result->type);
  return;
}

最后就导致了悬垂指针的出现，会出现一些未定义的行为。所以有时候崩溃，有时候不崩溃，尤其是后面调用可视化的接口更加有问题，在capi改动后c#的OCRResult和C的Result的那些参数应该也需要加上

ChaoII avatar Jul 10 '24 02:07 ChaoII

@ChaoII 你好，感谢你的回复，我大概也知道了这个是什么原因导致的，你说的这个c#的OCRResult的那些参数应该需要加上是怎么样在C# 里面添加？能力有限，不太清楚这部分怎么实现，需要您这边指导一下，万分感谢

1wang11lijian1 avatar Jul 10 '24 05:07 1wang11lijian1

你看c#源码里面的这一行OCRResult的定义是不是和C 的OCRResult字段对不上？

public class OCRResult {
  public List<int[]> boxes;   /// Member variable which indicates the coordinates of all detected target boxes in a single image. Each box is represented by 8 int values to indicate the 4 coordinates of the box, in the order of lower left, lower right, upper right, upper left.
  public List<string> text;  /// Member variable which indicates the content of the recognized text in multiple text boxes
  public List<float> rec_scores; ///  Member variable which indicates the confidence level of the recognized text.
  public List<float> cls_scores;  ///  Member variable which indicates the confidence level of the classification result of the text box
  public List<int> cls_labels;  /// Member variable which indicates the directional category of the textbox
  public ResultType type;

  public OCRResult() {
    this.boxes = new List<int[]>();
    this.text = new List<string>();
    this.rec_scores = new List<float>();
    this.cls_scores = new List<float>();
    this.cls_labels = new List<int>();
    this.type = ResultType.OCR;
  }

在c#的源码这一行定义了C#的OCRResult和C中的OCRResult的一些转换关系。

总体我觉得应该改三个地方，

BTW: 如果关注visual接口也需要注意CAPI中的这里需要补齐，空缺的。 C#那块的代码我也不太清楚。这一块问题的发现是我用rust绑定fastdeploy capi的过程中发现的。

ChaoII avatar Jul 10 '24 06:07 ChaoII

好的谢谢你 @ChaoII ，我来尝试一下，非常感谢！

1wang11lijian1 avatar Jul 12 '24 07:07 1wang11lijian1