RK3588S rknnlite版本1.4升级到1.5.2，yolov5推理速度变慢

[MHGL]

硬件：RK3588S 软件：

• rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl
• rknpu2

升级命令如下：

升级rknpu2
cp rknpu2/runtime/RK3588/Linux/rknn_server/aarch64/usr/bin/rknn_server /usr/bin/rknn_server
cp rknpu2/runtime/RK3588/Linux/librknn_api/aarch64/librknnrt.so /usr/lib/librknnrt.so
cp rknpu2/runtime/RK3588/Linux/librknn_api/aarch64/librknn_api.so /usr/lib/librknn_api.so

代码如下：

rknn_handle = RKNN(verbose=False)

    rknn_file = "weights/FastestDet.rknn"
    ret = rknn_handle.load_rknn(rknn_file)
    if ret:
        color_print("Failed load rknn model: {}".format(rknn_file), "red_bg")
        sys.exit(-1)

    ret = rknn_handle.init_runtime()
    if ret:
        color_print("Failed init rknn runtime!", "red_bg")
        sys.exit(-1)

    img_file = "data/0.jpg"
    cv_img = cv2.imread(img_file)
    color_print(">>> cv_img: {}".format(cv_img.shape))
    h0, w0, _ = cv_img.shape
    
    resize_img = cv2.resize(cv_img, (352, 352))

    for i in range(100):
        start_t = time.time()
        output = rknn_handle.inference([resize_img])[0]
        mid_t = time.time()
        print(">>> forward cost time: {:.3f}ms".format((mid_t-start_t)*1000))
        confidence, bbox, category = process(output, 0.65, w0, h0)
        index = nms_boxes(bbox, confidence, 0.45)
        end_t = time.time()
        color_print(">>> single frame: {} cost time: {:.3f}ms".format(resize_img.shape, (end_t-start_t)*1000))

如果在init_runtime中添加target="rk3588"参数，会报错如下:

librknn_api.so: undefined symbol: rknn_set_core_mask

如果不指定target参数，会正常运行，但是速度会慢2-3倍，请问是什么原因？

[dimentary]

Same issue, why we can't specify RK3588S as the target?

[daniilino]

I've got same here, please let me know if someone managed with it.

Thanks in advance

[besmaguesmi]

@MHGL @rudimytro @daniilino

Try to follow the instructions below:

1. Clone the rknpu2 repository: git clone https://github.com/rockchip-linux/rknpu2.git (if you didn't install it)
2. Copy the shared lib file to the lib dir: sudo cp rknpu2/runtime/RK3588/Linux/librknn_api/aarch64/librknnrt.so /usr/lib/librknnrt.so

This should solve your problem.

[dimentary]

@Guemann-ui thank you for your response, I did move the librknnrt.so file to /usr/lib/librknnrt.so, also tried adding it to LD_LIBRARY_PATH, but the issue is the same. Here is my full stack trace:

--> Init runtime environment
E Catch exception when init runtime!
E Traceback (most recent call last):
  File "/home/orangepi/.local/lib/python3.8/site-packages/rknnlite/api/rknn_lite.py", line 140, in init_runtime
    self.rknn_runtime = RKNNRuntime(root_dir=self.root_dir, target=target, device_id=device_id,
  File "rknnlite/api/rknn_runtime.py", line 319, in rknnlite.api.rknn_runtime.RKNNRuntime.__init__
  File "rknnlite/api/rknn_runtime.py", line 604, in rknnlite.api.rknn_runtime.RKNNRuntime._load_library
  File "/usr/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /home/orangepi/.local/lib/python3.8/site-packages/rknnlite/api/lib/hardware/DOLPHIN/linux-aarch64/librknn_api.so: undefined symbol: rknn_set_core_mask

Is it because I use 3588S not 3588? If I set target to 3588S I get:

Exception: Unsupported target: RK3588S! Currently, RKNN Toolkit Lite support these targets: ['RK3562', 'RK3566', 'RK3568', 'RK3588']

[besmaguesmi]

@rudimytro it could be… as RK3588S is not integrated as target and consider as RK3588, so try to replace it if you didn't yet

[dimentary]

I see, but if I put 3588 as a target I get undefined symbol: rknn_set_core_mask as I mentioned above

[besmaguesmi]

Did you install another version before the current one? also, did you use the same version to convert model and run inference? if you want, share your model and I will try on my board

[dimentary]

I used Ubuntu 20.04 with Python 3.8, both on the board and on the machine where I converted the model. When I do NOT specify the target it runs, but when I specify target as 3588 it gives me that error (same as for author of this Github issue). Here is the model: yolo.zip

[besmaguesmi]

@rudimytro did you specify the target as this (should be RK3588, not 3588):

if host_name == 'RK3588':
        ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0)

if you can share a piece of code where you get the issue, that would be helpful to check it

[bot-lin]

same issue here,
I am using RK3588s

[onepiece8971]

马上2024年7月了,这问题还是存在 RK3588s rknnlite2.0

[MHGL]

I RKNN: [13:27:50.940] RKNN Runtime Information, librknnrt version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11)
I RKNN: [13:27:50.940] RKNN Driver Information, version: 0.8.8
I RKNN: [13:27:50.940] RKNN Model Information, version: 6, toolkit version: 1.5.2+b642f30c(compiler version: 1.5.2 (c6b7b351a@2023-08-23T07:39:01)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
W RKNN: [13:27:50.955] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
>>> cv_img: (1080, 1920, 3)
>>> forward cost time: 37.469ms
>>> forward cost time: 35.757ms
>>> forward cost time: 34.492ms
>>> forward cost time: 40.367ms
>>> forward cost time: 34.305ms
>>> forward cost time: 40.115ms
>>> forward cost time: 33.643ms
>>> forward cost time: 39.148ms
>>> forward cost time: 33.534ms
>>> forward cost time: 39.168ms
>>> forward cost time: 33.555ms
>>> forward cost time: 40.385ms
>>> forward cost time: 34.389ms
>>> forward cost time: 34.492ms
>>> forward cost time: 40.153ms
>>> forward cost time: 34.355ms
>>> forward cost time: 40.361ms
>>> forward cost time: 34.375ms
>>> forward cost time: 40.104ms

试试0.8.8 + 1.6.0，yolov5s 640x640, 平均延时30多，RK3588J，未开性能模式，估计也差不了多少 也可试试C API，推理平均延时不到20

[XuanyueLee]

Same issue, why we can't specify RK3588S as the target?

In their official documents, it described that there only one parameter in init_runtime, and the parameter is core_mask. So the target parameter is prohibited, it only can be used in RKNN-Toolkit 2 but not RKNN-Toolkit Lite2. Hope that can help you!

02_Rockchip_RKNPU_User_Guide_RKNN_SDK_V2.0.0beta0_EN.pdf