Where can I find preprocessing and postprocessing code?

[aidevmin]

@marcoslucianops Where can I find preprocessing and postprocessing code in your repo? I want to know exactly how image is processed before put to model and NMS source code from raw output. Thanks.

[marcoslucianops]

The preprocessing is from DeepStream (NVIDIA) lib side. The postprocessing, in the YOLOv8 for example, is the Ultralytics postprocessing + the custom layer I added to have 3 outputs from their output: boxes [8400, 4], scores [8400, 1] (best score) and classes [8400, 1] (class id from the best score). After it, the DeepStream (NVIDIA) does the NMS. To disable the NMS, you should set cluster-mode=4 in the config_infer_primary file (according to your model).

[aidevmin]

@marcoslucianops thanks, custom layer you added is nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp?

[marcoslucianops]

The custom layer is in the exporter files (ONNX) in the utils folder. For the Darknet models, ou can see the output in nvdsinfer_custom_impl_Yolo/yoloForward.cu, nvdsinfer_custom_impl_Yolo/yoloForward_nc.cu and nvdsinfer_custom_impl_Yolo/yoloForward_v2.cu files. The _nc means models with new_coords=1 in the cfg file.

[aidevmin]

@marcoslucianops thanks for quick and detailed response

[aidevmin]

@marcoslucianops I have 2 questions:

1. Where I can find NMS of Nvidia?
2. If my model includes NMS, as you mentioned I need to set cluster_mode=4, but what the happen with the file nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp and parametwr nvdsparserbbox in config file? As my underatanding, bbox parser convert output to specific format before passing in NMS code. Thanks a lot.

[marcoslucianops]

1. Where I can find NMS of Nvidia?

/opt/nvidia/deepstream/deepstream-6.2/sources/gst-plugins/gst-nvdspostprocess/postprocesslib_impl/post_processor_detect.cpp lines 540-591.

2. If my model includes NMS, as you mentioned I need to set cluster_mode=4, but what the happen with the file nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp and parametwr nvdsparserbbox in config file? As my underatanding, bbox parser convert output to specific format before passing in NMS code.

The NMS is did after the nvdsparsebbox function.