Inference time based on Hardware (GPU)

[Dhiaeddine-Oussayed]

I have tested yolov7 vs yolov5 inference time and I got these results:

60ms yolov5 vs 15ms yolov7 : Nvidia V100 16Gb 20ms yolov5 vs 20ms yolov7 : Nvidia Jetson AGX Orin 32Gb 15ms yolov5 vs 30ms yolov7 : Nvidia GTX 1080 8Gb

Does anyone have an idea on why I am having this weird behavior ? (All not accelerated)

[AlexeyAB]

What commands did you use to measure inference speed?

[mkhoshbin72]

I had same experience with NVIDIA 1080 and 3070. On NVIDIA 3070, YOLOv7 is faster than YOLOv5 but on NVIDIA 1080 it's the opposite.

[Dhiaeddine-Oussayed]

@AlexeyAB I just used the time difference using time_synchronized() in the detect.py script.

t1 = time_synchronized()
pred = model(img, augment=opt.augment)[0]
t2 = time_synchronized()
inference_time = t2 - t1

[AlexeyAB]

Which models did you compare?

YOLOv7 - 161 FPS, 51.2% AP https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt and YOLOv5m6 - 90 FPS, 51.3% AP https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5m6.pt ?

Did you compare both models without NMS? What float precision did you use?

[Dhiaeddine-Oussayed]

@AlexeyAB Yes, I used YOLOV7 that you mentioned but YOLOV5m 45.4% AP.

I compared with NMS, I used everything as default. I did not specify any float precision. I've just ran detect.py on same test case. this is the command I used for training both models: YOLOv7: python train.py --data path/to/data --weights training_yolov7.pt --epochs 50 --batch 8 --cfg yolov7.yaml --hyp hyp.scratch.custom.yaml YOLOv5: python train.py --data path/to/data --weights yolov5m.pt --epochs 50 --batch 8 --cfg yolov5m.yaml --hyp hyp.custom.yaml

[AlexeyAB]

I mean what command did you use to measure latency for Inference?

So for YOLOv5m 45.4% AP vs YOLOv7 51.2% AP you got: 60ms yolov5m vs 15ms yolov7 : Nvidia V100 16Gb 20ms yolov5m vs 20ms yolov7 : Nvidia Jetson AGX Orin 32Gb 15ms yolov5m vs 30ms yolov7 : Nvidia GTX 1080 8Gb

For all these GPUs YOLOv7 is better in terms of speed and accuracy, since

• YOLOv7 51.2% AP is faster than YOLOv5m 45.4% AP on V100
• YOLOv7 51.2% AP is more +6% AP more accurate with the same speed as YOLOv5m 45.4% AP on Jetson AGX Orin
• YOLOv7 51.2% AP is much faster than YOLOv5m6 51.3% AP even on GTX 1080

In general YOLOv7 parallelizes much better than YOLOv5, so the faster and newer the GPU, the more preferable YOLOv7 will be.

[Dhiaeddine-Oussayed]

@AlexeyAB that's what I am expecting from YOLOv7. I did not use any command I just read what this line from the detect.py script prints

[damvantai]

YOLOv7 in rtx3090, img-size 640 Done. (4.7ms) Inference, (0.4ms) NMS