DeepStream-Yolo
DeepStream-Yolo copied to clipboard
marcoslucianops
YoloNAS no detections after conversion
To recreate the issue:
Step 1: Trained Yolo NAS model and got a ckpt_best.pth model file. (Tested this model on several videos and it works perfectly).
Step 2: Followed the steps outlined in the README to convert to ONNX. I trained a custom model with 2 classes only and I trained it with the large yolo nas model, so I ran the following command to convert my model:
python3 export_yolonas.py -m yolo_nas_l -w yolo_nas_l.pth --dynamic --classes 2
Step 3: I ran the deepstream-app with the proper config file pointing to the plugin. It successfully converted the .onnx file converted from above to a .engine file and started running on the video. However, I'm not seeing any bounding boxes on the video at all. I tried converting to FP16, FP32 and INT8(All of which I have done with success with YoloV7).
Please see attach my config files(config_infer_primary_yolo-nas.txt and deepstreawm_app_config_test.txt) for reference: config_infer_primary_yolo-nas.txt deepstream_app_config_test.txt
Any ideas?
I added the config_infer_primary_yolonas_custom.txt file to the repo. For custom YOLO-NAS model you should use it.
The pretrained YOLO-NAS is using net-scale-factor=0.0039215697906911373 and the custom YOLO-NAS is trained with net-scale-factor=1.
Thank you!
How can I determine the net-scale-factor of a particular model? For example, if I convert the model to use INT8, then the detections disappear and I used 2,000 images from validation to run the calibrations.
The net-scale-factor and offsets keys are related to image normalization in the image preprocessing.
The net-scale-factor is related to std values (example for std = 0.229, 0.224, 0.225:
net_scale_fator = 1 / (np.array([0.229, 0.224, 0.225]).mean() * 255)
print(net_scale_fator)
The offsets is related to mean values (example for mean = 0.485, 0.456, 0.406:
offsets = np.array([0.485, 0.456, 0.406]) * 255
print(";".join(str(x) for x in offsets))
About the INT8 model, I need to check later how to export it correctly to use in the DeepStream. For now it should not work.
Hi @marcoslucianops ,
I trained my custom YOLO-NAS model using https://github.com/naseemap47/YOLO-NAS
where I will find my model std
I am not giving any std input value.
I only given input size as 640x640
@naseemap-er try both config files available for YOLO-NAS and check which one you will get detections.
@marcoslucianops
When I used coco net-scale-factor, its working for me.
But I am getting few false detections on the sides of frame.
But when I use YOLO-NAS inference. I didn't get that kind of detections.
When I try with net-scale-factor = 1.
I am getting full false predictions.
So I thinking it's due to net-scale-factor.
But to find my net-scale-factor, I need std, according your formula.
So can you help me to find the std value from training.
If you didn't change it in the training, it should be the default values.
@marcoslucianops thanks for your support.
Can you tell me when will be the Deepstream YOLOv9 will release ?
It will very helpful.