PhotonVision
Populate OD models into images
Instead of shipping OD models with the jar, which gets kinda bulky, we plop them into each image during the CI build.
I need to ponder some more, cause then we can't extract from the jar or clear models and restore and whatnot. There might be potential for packing different models into the jar based on the platform we're building for, or storing everything in a .zip at least so it's smaller. Anyways, pondering.......
Pondering some more, I think we could just make a defaultModels dir, and then that allows us to still extract from there and clear stuff. Also consider how shipping new models will work with labels.
Okie dokie pondered even more, and here's what we got.
We make a dir in each image: /opt/photonvision/modelsImmutable. This dir has r/w 111. Instead of extracting from the jar in code, we extract from here. Warn if no models are found here. Warn if perms are not 111.
In the monorepo, models get moved into a top-level folder, models. This top-level folder will contain subfolders for each platform supporting OD, named using our typical conventions. These folders will contain the various models for each platform.
We will also publish models to our releases because they will no longer be shipped with the jar, meaning teams who want an updated model need to upload it manually or reflash. The only models that will be published will be models for the current season. This will be accomplished by creating subfolders under each platform directory, named with the season's year. All models in the subfolders named with the year listed in the release will be published. This process will not be recursive, so models intended for that year, but that have become outdated, can be placed in a deprecated folder inside the year folder.
Example Monorepo Directory Structure
photonvision/
└── models/
├── rubik/
│ ├── 2024/
│ │ ├── note-detection-v1.tflite
│ │ ├── note-detection-v2.tflite
│ │ └── deprecated/
│ │ └── note-detection-v0.tflite
│ ├── 2025/
│ │ ├── coral-detection-v1.tflite
│ │ └── algae-detection-v1.tflite
│ └── general/
| ├── coco.tflite
│ └── apriltag-detection.tflite
│
└── opi/
├── 2024/
│ ├── note-detection-v1.rknn
│ └── note-detection-v2.rknn
├── 2025/
│ ├── coral-detection-v1.rknn
│ └── algae-detection-v1.rknn
└── general/
├── coco.rknn
└── apriltag-detection.rknn
The top-level folder in the monorepo will also contain a script (similar to armrunner.sh), that will check the device-tree of the image and copy the appropriate models. The script should print the device it's identified, and all models being copied. This script should also create the dir mentioned above -- /opt/photonvision/modelsImmutable -- and set the permissions to 111 after copying the models in. It will be ran in CI after armrunner.sh.
Assuming the image size is flat, or at least close to flat? Just dont' wanna blast past github's limits, or double the user download/flash time.
As long as the docs talk about how to fully back-up the image currently running (download all models AND settings, reflash, then restore models and settings) I think I'm down with this as a general idea?
Assuming the image size is flat, or at least close to flat? Just dont' wanna blast past github's limits, or double the user download/flash time.
It'll add a whole like 20 megs (the horror!). For the images, that's not a ton, but we'll drop about 30 MB from the jar, which is a lot.
As long as the docs talk about how to fully back-up the image currently running (download all models AND settings, reflash, then restore models and settings) I think I'm down with this as a general idea?
Yea all of that should be already documented.
Do we add this to the install script?
This would be done in CI in the monorepo, similar to how we copy the new jar into the image. This would, I suppose, mean models aren't going to be included in the images from the image-modifier-repo
That means that anyone who runs the installer directly also won’t get the default models.
That means that anyone who runs the installer directly also won’t get the default models.
The default models are only meant for platforms that we officially support and ship an image for. I wouldn't expect anyone to run an installer on these platforms, instead of flashing an image. I'd almost view this as a benefit, as these people wouldn't be able to use the default models, and thus shouldn't have to download something useless.
Also to provide some context, models are currently 18% of the jar's file size.