keras-cv
keras-cv copied to clipboard
keras-team
Segment Anything - Next Steps
Segment Anything model in KerasCV
#1987 added the Segment Anything model to KerasCV but it needs a guide to demonstrate training and prompting the trained model. It would also be nice to have some benchmarks for it. Following is a list of tasks that still need to be addressed:
- [x] Writing a guide to use the model (similar to the predictor demo on the original repo)
- [ ] Writing a guide to generate masks for the entire image (similar to the automatic mask generation demo on the original repo)
- [ ] (@IMvision12) A guide for end-to-end model training. See the appendix "A. Segment Anything Model and Task Details" from the paper that explains the training method used for the existing presets.
- [ ] Adding support for text prompts (e.g. CLIP). The paper mentions that a CLIP model was used to encode text prompts.
- [x] (Optionally) It'd also be nice to have benchmarks for all the backends and document it somewhere.
@tirthasheshpatel I can take the Writing a guide to use the model
@tirthasheshpatel I can take up "Writing a guide to generate masks for the entire image". Thank you.
@tirthasheshpatel I can take up "Writing a guide to generate masks for the entire image". Thank you.
@tirthasheshpatel Since this guide requires implementation of predictor and maskgenerator, I am working on this.
@tirthasheshpatel Since the predictor and automaticmaskgenerator work is in wip and almost done by you as I checked your repo here: https://github.com/tirthasheshpatel/segment_anything_keras/tree/keras-cv-update. If there is any help required, I would be happy to help. Thank you.