unusual high time for ex.extract() line 345-350 in inference.py Yolov8

[manish33scss]

   begin = perf_counter()
    ret1, mat_out1 = ex.extract(output_names[0])  # stride 8
    assert not ret1, f'extract {output_names[0]} with something wrong!'
    ret2, mat_out2 = ex.extract(output_names[1])  # stride 16
    assert not ret2, f'extract {output_names[1]} with something wrong!'
    ret3, mat_out3 = ex.extract(output_names[2])  # stride 32
    assert not ret3, f'extract {output_names[2]} with something wrong!'
   
     end = perf_counter()

These lines of code are taking around 400-500ms for each frame, running this on jetson nano !! What could be the problem ? any work around ?

[triple-Mu]

How about adding:

net.opt.use_vulkan_compute = True
net.opt.num_threads = 4
net.opt.use_bf16_storage = True

Or convert your ncnn with int8 quant.

[manish33scss]

still same results, even the c++ file gives 1 fps on jetson nano !

[triple-Mu]

still same results, even the c++ file gives 1 fps on jetson nano !

Do you try int8 quant？ Or you can try my yolov8-tensorrt