YOLOv7 vs YOLOv5 reproducible comparison of accuracy and speed

[AlexeyAB]

When tested in an identical environment on a nVidia T4 GPU:

1. YOLOv7 (51.2% AP, 12.6ms) is 1.5x times faster and +6.3% AP more accurate than YOLOv5s6 (44.9% AP, 18.7ms)
2. YOLOv7 (51.2% AP, 12.6ms) has almost the same accuracy but 4x times faster than YOLOv5m6 (51.3% AP, 49.1ms)
3. YOLOv7-e6 1280x1280 (55.9% AP, 46.6ms) is +0.9% AP more accurate and 4x times faster than YOLOv5x6 1280x1280 (55.0% AP, 188.1ms)

---

YOLOv7 (51.2% AP, 12.6ms) is 1.5x times faster and +6.3% AP more accurate than YOLOv5s6 (44.9% AP, 18.7ms)

https://colab.research.google.com/github/WongKinYiu/yolov7/blob/main/tools/compare_YOLOv7_vs_YOLOv5s6.ipynb

!python test.py --data data/coco.yaml --img 640 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov7.pt --name yolov7_640_val
...
Speed: 12.6/0.9/13.5 ms inference/NMS/total per 640x640 image at batch-size 1
...
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.512

!python val.py --data data/coco.yaml --img 1280 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov5s6.pt --name yolov5s6_1280_val
...
Speed: 0.7ms pre-process, 18.7ms inference, 1.7ms NMS per image at shape (1, 3, 1280, 1280)
...
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.449

---

YOLOv7 (51.2% AP, 12.6ms) has almost the same accuracy but 4x times faster than YOLOv5m6 (51.3% AP, 49.1ms)

https://colab.research.google.com/github/WongKinYiu/yolov7/blob/main/tools/compare_YOLOv7_vs_YOLOv5m6.ipynb

!python test.py --data data/coco.yaml --img 640 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov7.pt --name yolov7_640_val
...
Speed: 12.6/0.9/13.5 ms inference/NMS/total per 640x640 image at batch-size 1
...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.512

!python val.py --data data/coco.yaml --img 1280 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov5m6.pt --name yolov5m6_1280_val
...
Speed: 0.6ms pre-process, 49.1ms inference, 1.7ms NMS per image at shape (1, 3, 1280, 1280)
...
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.513

---

YOLOv7-e6 1280x1280 (55.9% AP, 46.6ms) is +0.9% AP more accurate and 4x times faster than YOLOv5x6 1280x1280 (55.0% AP, 188.1ms)

https://colab.research.google.com/github/WongKinYiu/yolov7/blob/main/tools/compare_YOLOv7e6_vs_YOLOv5x6.ipynb

!python test.py --data data/coco.yaml --img 1280 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov7-e6.pt --name yolov7_e6_1280_val
...
Speed: 46.6/1.5/48.1 ms inference/NMS/total per 1280x1280 image at batch-size 1
...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.559

!python val.py --data data/coco.yaml --img 1280 --batch 1 --conf 0.001 --iou 0.65 --device 0 --weights yolov5x6.pt --name yolov5x6_1280_val
...
Speed: 0.7ms pre-process, 188.1ms inference, 1.8ms NMS per image at shape (1, 3, 1280, 1280)
...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.550

[AlexeyAB]

Paperswithcode provides a comparison to other networks with references to values in papers: https://paperswithcode.com/sota/real-time-object-detection-on-coco?dimension=FPS%20%28V100%2C%20b%3D1%29

[AlexeyAB]

[spacewalk01]

but by default, yolov7 uses half-precision setting for test while yolov5 uses fp32 precision. https://github.com/WongKinYiu/yolov7/blob/b8956dd5a5bcbb81c92944545ca03390c22a695f/test.py#L40

[Dhiaeddine-Oussayed]

I have ran inference tests with both yolov7 and yolov5 on a Jetson AGX Orin 32Gb, but they both had the same inference time (20ms, in my case). While when I tested with same data on an Amazon instance yolov5 was aound 60ms and yolov7 was around 15ms. Any idea why yolov7 not performing better on the Jetson AGX Orin ?

[Dhiaeddine-Oussayed]

@AlexeyAB any idea about issue just above ?