Expected performance on an Jetson Orin AGX

[AWilco]

Search before asking

• [X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

We have been using your YoloV5 model for a number of years, it provides impressive detection performance and the documentation is very thorough.

We have recently moved to testing our models on the Jetson Orin AGX unit. We haven't seen an improvement in performance compared to the Xavier AGX unit that we expected. With the YoloV5s model running against a 640x640 image we are seeing inference times of 10ms/image on both units (FP32 or FP16, batch size 1). A (blog post)[https://www.stereolabs.com/blog/performance-of-yolo-v5-v7-and-v8] gives performance that should be approaching 3ms/image, although they don't specify FP or INT8, or their batch size.

Do you have any benchmarks available on the Orin AGX unit? I'm looking to try and understand if there is some setting or optimisation I have missed, or whether it is expected that the YoloV5s model wouldn't see any speed improvement between the Xavier and Orin AGX units.

I appreciate that the Yolov5 model is not the current model and we are looking to move to YoloV7 or YoloV8. If you have benchmarks for these that would also be useful.

Thanks for your time.

Additional

No response

[glenn-jocher]

@AWilco hello!

Thank you for reaching out and for your ongoing support of YOLOv5! It's fantastic to hear about your experiences and your move towards testing on newer hardware like the Jetson Orin AGX. 🚀

To address your question, currently, we do not have specific benchmarks available for YOLOv5 on the Jetson Orin AGX. Performance can indeed vary based on numerous factors including model version, image resolution, inference precision (FP32, FP16, INT8), and additional optimizations.

For the Jetson Orin AGX, it's essential to ensure you are leveraging all available optimizations. This could include TensorRT for optimizing the model, experimenting with different precision modes (with a close eye on FP16 and INT8 if your application permits), and tuning your deployment setup to the specific hardware capabilities of the Orin AGX.

Since you're considering moving to newer versions, I'd recommend staying tuned to our updates and the community discussions, as benchmarks and optimization recommendations for platforms like the Orin AGX are likely to evolve. Keep in mind, when moving towards other models or versions, to review their respective repositories for the most accurate and up-to-date information.

Your feedback regarding the need for specific benchmarks on hardware like the Orin AGX is valuable, and we'll certainly take it into consideration for our future updates.

Thank you for your contribution to the YOLO community, and we look forward to assisting you further as you explore newer models and optimization techniques. 🌟

Warm

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[AWilco]

Hi Glenn,

Sorry for not replying sooner. Thanks for your feedback. We've continued our work on the Orin and are also now trying YoloV8, which does look to give an improvement for us.

[glenn-jocher]

Hello,

Great to hear that you're seeing improvements with YOLOv8 on the Orin! If you need further assistance or have specific questions as you continue your work, feel free to reach out. Happy detecting! 😊