rf-detr
rf-detr copied to clipboard
roboflow
🚀 Question & Guidance: Integrating YOLOv8 Heads & Training from Scratch on COCO
Hi team, first off, amazing work on this project!
I'm currently exploring the possibility of extending the model's capabilities by integrating the highly efficient detection and segmentation heads from YOLOv8. My goal is to leverage your powerful backbone with the proven performance of YOLOv8's prediction layers.
1. Hybrid Model with YOLOv8 Heads My plan is to take the feature maps from the backbone and feed them into the YOLOv8 Detect and Segment heads. I have a few architectural questions and would appreciate any guidance:
- Feature Map Alignment: What are your recommendations for adapting the backbone's output feature maps to match the channel/size requirements of the YOLOv8 heads? Would adding a custom neck or a few convolutional layers be the best approach?
2. Training from Scratch on COCO To establish a baseline, I first tried training your medium-sized model on the COCO dataset from scratch (without pre-trained weights). I ran the training for 30 epochs.The mAP score remained flat and showed no signs of improvement. It seems the model isn't learning effectively.Could you share any insights or recommended settings for training on COCO from scratch?
Is your goal to get segmentation working?
When you say training from scratch, do you also mean no dinov2 weights?
Yes, I want to get it working for instance segmentation.When I say training from scratch is using the dinov2 weights but without the detection weights.
got it. we'll have a segmentation head out soon, actively working on it. yolo series heads aren't directly applicable to detr series models so we've built our own