keras-cv
keras-cv copied to clipboard
Add MaskRCNN model as a test for the wrapper approach with TF model garden code
Note that this PR is more for a demo and not intend to be submitted at the moment.
What does this PR do?
Add MaskRCNN model as a test for the wrapper approach with TF model garden code.
- The current class only contains the model building logic.
- Most of the tunable params are currently backed in the function, and not exposed to use yet.
- Verify the model building logic in the test. (will need tfm pip deps to work)
Next step:
- Configure the input parsing logic for coco, and also anchor generation.
- Mimic the training loop logic as model.train_step/eval_step
- Train model e2e.
Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the contributor guideline, Pull Request section?
- [ ] Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
- [x] Did you write any new necessary tests?
- [ ] If this adds a new model, can you run a few training steps on TPU in Colab to ensure that no XLA incompatible OP are used?
Who can review?
Comments from the #753:
- We should raise model specific error message when TFM is not installed.
- The model class interface should be adjusted to follow the Keras CV format.
- The function approach should be changed to the model class approach, which has a delegate TFM MaskRCNN in the class body.
Relevant ticket. https://github.com/keras-team/keras-cv/issues/623
Open questions:
- how should loss configuration work ? There are three losses in MaskRCNN: classification loss, box regression loss and mask loss. An additional complexity id that masks are predicted per class but only the mask loss of the predicted class is taken into account.
- metric configuration: this should be compatible with KerasCV's COCO metrics.
- support for KerasCV-style bounding box formats.
- TPU support ?
- can data encoding/decoding use KerasCV APIs?
Note: the goal here is to wrap the Model Graden model but not the Model Garden data pipeline code. Users can use Keras preprocessing layers to format a data feed appropriately and we will provide example code in the accompanying templates.
Should we close this as we most likely will add MaskRCNN on top of existing FasterRCNN?
Should we close this as we most likely will add MaskRCNN on top of existing FasterRCNN?
yeah I think so.
Discussed offline, it seems we all agree to close it for now.