Quantization with Pytorch

[katia-katkat]

Search before asking

• [X] I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

No response

Bug

Hello,

I am trying to quantize yolov5 trained with a custum dataset with Pytorch, however, when running the following line of code :

model_prepared = quantize_fx.prepare_fx(model_to_quantize, qconfig_mapping, example_inputs)

I got a TraceError : symbolically traced variables cannot be used as inputs to control flow

I checked that qconfig_mapping and example_inputs weren't causing the issue so the only one left is "model_to_quantize" So my guess is in the way I am loading the model : model_fp = torch.hub.load('ultralytics/yolov5', 'custom', path='./yolov5s', force_reload=True) I want to know what is the correct way to load the model so that it is compatible with the format quantize_fx.prepare_fx function expects?

Thank you in advance !!

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• [ ] Yes I'd like to help by submitting a PR!

[glenn-jocher]

@katia-katkat hello!

Thank you for reaching out with your quantization question. It seems like you're encountering a trace error during the symbolic tracing step required for FX graph mode quantization in PyTorch.

When you load the model using torch.hub.load, it should be compatible with the quantization process. However, the error you're seeing might be related to dynamic control flow within the YOLOv5 model, which can be challenging for the symbolic tracer.

To ensure compatibility, you might need to make modifications to the model's forward method to replace dynamic control flow with static alternatives where possible. This can be quite involved and may require an in-depth understanding of the model's architecture.

For more detailed guidance on quantization and potential workarounds, I recommend checking out the PyTorch quantization documentation and the YOLOv5 documentation at our official Ultralytics Docs page.

If you continue to face issues, please provide a more detailed code snippet and the full error traceback, so we can better understand the context and assist you further.

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐