Model Ensembling With PyTorch Hub

[ertugrul-dmr]

Search before asking

• [X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I know it's possible to ensemble multiple models with detect.py, but is it possible to do ensemble or do TTA with PyTorch Hub interface?

Let's say:

import torch

# Model
model_a = torch.hub.load('path/to/yolov5', 'custom', path='path/to/model_1.pt', source='local')
model_b = torch.hub.load('path/to/yolov5', 'custom', path='path/to/model_2.pt', source='local')

# Image
im = 'https://ultralytics.com/images/zidane.jpg'

# Inference
results = ???

results.pandas().xyxy[0]

How could I get results in a standardized way with this setting?

Additional

No response

[glenn-jocher]

@ertugrul-dmr YOLOv5 PyTorch Hub models can use TTA with the augment argument:

results = model(im, augment=True)

You could customize some sort of Ensemble() using the Ensemble() module on your own, i.e. Ensemble two models and then wrap them in AutoShape: https://github.com/ultralytics/yolov5/blob/92e47b85d952274480c8c5efa5900e686241a96b/models/experimental.py#L61-L72

[kaamlaS]

@ertugrul-dmr Hi, did you find a way to ensemble with torch.hub?

[ertugrul-dmr]

@kaamlaS yeah, wasn't an optimal solution but I kept results for every single image and then applied Weighted boxes fusion over them.

Ref: Weighted boxes fusion

[kaamlaS]

okay, cool. im going to give this a look. thank you!

[Sarveshsh8]

did you find any location for ensembling using torch.hub.load?

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[glenn-jocher]

@Sarveshsh8 Weighted boxes fusion seems like a solid approach for ensembling. Thanks for sharing! Let me know if there's anything else I can help with.