roboflow
Rotated Bounding Boxes
Is there a plan to support oriented bounding boxes in a future version? It would be extremely beneficial for applications in aerial imaging.
Thank you very much for your excellent work!
there's no plan right now but noted, we'll keep in mind that there's interest! :) can you share a bit more about your use case? would segmentation solve?
Thanks @isaacrob-roboflow, are there plans for segmentation? (It would probably be more useful in the case I was considering).
Thank you for you prompt reply @isaacrob-roboflow , yes there are quite a few use cases i can show you such as this image coming from DOTA dataset paper showcasing OBB superior clarity in particular in the context of aerial images (drones, satellites etc).
It became a topic also recently on Torchvision repo but as far as i know just a few steps were taken in that direction. Unfortunately segmentation can only partially provide a solution and the requirement of building segmentation masks to train a DL model are often unfeasible.
I'm generally pretty pro obb, big downside in my mind is we don't have obb pretraining datasets. but it's possible to 'monkey patch' tooling for it into the model. supporting obb is a big discussion inside roboflow right now generally
your thoughts are super useful in informing our direction!
what dataset format do you use to support obb training? can you talk a bit about your current pipeline?
Currently working with (center x, center y, width, height, orientation) format. Not many fancy things in my current pipeline, I'm working with a two stage detector hence the "complex" part lies in computing rotated IOU efficiently both from memory and speed POV. Representing BBOXs in other format is also completely fine and the conversion to 4 corner coordinates for example is straightforward. As far as i know DOTA is the most comprehensive datasets with OBB and i saw that it is used also in YOLO models with OBBs.
If OBB support is added, will a 360deg option be added? There are cases where this is needed, but as far as I can tell things like yolo don’t support this only 180deg. Even things like predicting bboxes that extend outside the image.
In my opinion 4 point polygon is more useful than the standard rotated bounding boxes because you can handle perspective for objects. OBB are a then a subset of this in the 360deg case. Although not the 180deg case.
@nmichlo do you know why those other frameworks only support 180deg?
@isaacrob-roboflow I'm guessing here, but I think the main reason is if you want a minimal bounding box vs a truly oriented bounding box. 180° (edit: or 90? or maybe even 45°) will get you a minimal one, 360 will get you a truly oriented one. Useful in different scenarios, could hurt performance if there isn't an option, but most users might not be concerned with this.
EDIT: its possible it was actually the data pre-processing the frameworks apply, maybe they were sorting the points instead of respecting point order -- It would increase performance on some datasets.
EDIT2: in @alecenta's image, the OBB boxes probably are all just the minimal ones, but what if you actually also want to predict which way's the trucks are facing -- heuristics like aspect ratio aren't reliable.
Thinking more about this, I definitely still think an xyxyxyxy (4 corners) implementation of OBB would be more useful than xywhr -- all cases in my previous comment could be covered with the former, with the latter being a subset. It then just becomes a dataset preparation task to handle truly oriented OBB vs minimal OBB, and in different input formats too, can easily convert between them.
Furthermore, xyxyxyxy is akin to a 4 point version of instance segmentation, so if you actually implement segmentation you've effectively generalised the training pipeline over OBB too, if you just limit the inputs to 4 points.
@isaacrob-roboflow have there been updates internally about if and when you’ll ship OBB?
@isaacrob-roboflow any updates on an ETA for when you will ship with OBB?
Hi! We'll ship it once Roboflow supports OBB natively. It's not on our timeline right now.
I have been redirected to this issue from a keypoint detection question -- how about keypoints? Roboflow supports those already in other contexts. Any plans for this? Thanks!
We do plan on supporting keypoints :) on the list but we have some bigger surprises coming soon
