PhotonVision
Implement YoloV11 OBB detection for RKNN JNI
Description
Implemented YoloV11 Oriented Bounding Box detection so recognized bounding boxes also include an angle theta describing their rotation. Helpful when the overall axis aligned BB is too inaccurate or big. Only supports RKNN JNI for now. (Corresponding PR: https://github.com/PhotonVision/rknn_jni/pull/28)
Meta
Merge checklist:
- [x] Pull Request title is short, imperative summary of proposed changes
- [x] The description documents the what and why
- [x] If this PR changes behavior or adds a feature, user documentation is updated
- [ ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
- [x] If this PR touches configuration, this is backwards compatible with settings back to v2025.3.2
- [ done ] Pull Request title is short, imperative summary of proposed changes
- [ done ] The description documents the what and why
- [ done ] If this PR changes behavior or adds a feature, user documentation is updated
- [ not sure ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
- [ done ] If this PR touches configuration, this is backwards compatible with settings back to v2025.3.2
For future reference, if you put an x in between the bracket, it marks the checkbox as completed. See below for an example. You can also leave it as is, and just check the box in the Web UI.
- [ ] not complete
- [x] completed
- [ ] not complete
- [x] completed
@james-kwong I'm having some issues getting this to work on my opi, is there anything else you did to get this to work?
@samfreund
What are the errors you're getting when trying to run the code?
On my machine it works out of the box. Perhaps it is a difference with the .rknn models? I tested it with the cage .rknn that should be pushed with this PR.
I think there was a very specific process I used when converting the RKNNs
@samfreund
What are the errors you're getting when trying to run the code?
On my machine it works out of the box. Perhaps it is a difference with the .rknn models? I tested it with the cage .rknn that should be pushed with this PR
I'm guessing it's coming from my attempts to build the rknn_jni and publish it locally. Can you share how you're doing that, and what hardware you're on?
Oh shoot, wait I forgot to change the rknn jni version , so it probably isn't picking it up
Oh shoot, wait I forgot to change the rknn jni version , so it probably isn't picking it up
That's not the issue, as the rknn_jni version isn't published yet as it hasn't been merged. I'm trying to test locally, so that it can get merged, but I'm having difficulties publishing it locally.
Also, if you're willing to do a writeup on the steps you took to convert the OBB model and how they differ from the typical detection model, that would be really helpful.
Wait by the way, thank you for all your help in fixing some of the formatting and stuff up.
I'm currently at university so I have to wait until I get back home before I can access the Orange PI. I noticed how it draws both the axis-aligned BB and OBB, but I didn't add the axis aligned one. It seems how Photonvision works is whenever its drawing a contour, it overlays the minimum rectangle over it as well.
Do you have any ideas on how I could fix that ?
Wait by the way, thank you for all your help in fixing some of the formatting and stuff up.
In the future, you can do it yourself!
I'm currently at university so I have to wait until I get back home before I can access the Orange PI. I noticed how it draws both the axis-aligned BB and OBB, but I didn't add the axis aligned one. It seems how Photonvision works is whenever its drawing a contour, it overlays the minimum rectangle over it as well.
Do you have any ideas on how I could fix that ?
Drawing 2d targets is done here, I would guess that's where you would fix it.