安装官方文档使用FastDeploy部署OCR识别V3的版本，不稳定，在用不同尺寸的图片的时候就经常出问题

Open xiulianzw opened this issue 2 years ago • 2 comments

环境

【FastDeploy版本】：镜像registry.baidubce.com/paddlepaddle/fastdeploy:1.0.4-gpu-cuda11.4-trt8.5-21.10
【启动命令】fastdeployserver --model-repository=/ocr_serving/models
【进入容器】 docker exec -it -u root fastdeploy bash
【系统平台】: Linux x64(Ubuntu 16.04)
【硬件】： Nvidia GPU 2070 super， CUDA 12，NVIDA driver 525.89.02

【bug】
- FastDeploy启动成功之后使用client.py测试OCR的识别结果发现，如果一直是同一尺寸的图片貌似没问题，但是如果调用的过程中换其他的图片就会报错，有时候报显存不足，有时候识别的时候box报index out of range，输出box看的时候发现是空的 I0901 03:58:34.384452 147 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001 I0901 03:58:34.385091 147 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000 I0901 03:58:34.426893 147 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002 W0901 03:58:45.404812 147 memory.cc:135] Failed to allocate CUDA memory with byte size 81885000 on GPU 0: CNMEM_STATUS_OUT_OF_MEMORY, falling back to pinned system memory 0901 03:58:45.406278 226 pb_stub.cc:402] Failed to process the request(s) for model 'det_postprocess_0', message: TritonModelException: in ensemble 'rec_pp', softmax_5.tmp_0: failed to perform CUDA copy: invalid argument

At: /ocr_serving/models/det_postprocess/1/model.py(206): execute

Sep 01 '23 05:09 xiulianzw

已收到问题反馈

Feb 06 '24 08:02 rainyfly