How should I extract segmentation ontology from annotations?

[xiao7199]

Hi,

Thanks for your excellent work and It's awesom to have segment ontology annotations. But how should I extract the parent-children relationship from the segmentation annotations? BTW, do you know if there is another dataset having ontology annotations.

Thanks

[KMnP]

Hi, could you elaborate on "extract the parent-children relationship from the segmentation annotations"? Not sure if I fully get what you mean. Cuz you can get the super-class of the attributes and categories. But what do you mean by segmentation annotations?

[amrahsmaytas]

Hi, could you elaborate on "extract the parent-children relationship from the segmentation annotations"? Not sure if I fully get what you mean. Cuz you can get the super-class of the attributes and categories. But what do you mean by segmentation annotations?

Hi @KMnP, can you brief it in more detail about it?
Currently, I have trained a model to predict classes/categories from an image, but I want to predict attributes also from the same image, I am unable to understand the data format, currently, can you please guide me through, how to predict the attributes or extract the attribute by using the supercategory/level/any relationship between category and attributes?

[KMnP]

hi @amrahsmaytas , I think your question is how to get attribute from category.

We separate the category and attribute annotation process. So attributes are another kind of label for a certain region in an image. So you should treat attributes as another task .

You can check out our Attribute-Mask RCNN model, which it perform both detection + attribute recognition tasks.

[amrahsmaytas]

hi @KMnP, thanks for your reply, please have a check on the below-mentioned points:

Done

• Trained Detectron2 on categories with no Augmentation, No image pre/post-processing and was able to achieve an AP ~32

Have to Do

• Have to know, whats the Good AP, achieved so far on the fashionpedia dataset regarding categories
• understand the attributes annotation process (can i train it separately, considering it as an instance segmentation problem with 294 attributes as classes)
• Training Detectron2 on attributes, so that i can predict attributes from an image, independent of categories in image
• where to find the Attribute-MASK RCNN model?
• To get clarity on, whether can i input any google image into fashionpedia API and expect the results to be classes + attributes from a single image? If yes, Please guide me through it!

Please be kind to guide me through the above-mentioned points, Thanks!

[KMnP]

Hi @amrahsmaytas

"Have to know, whats the Good AP, achieved so far on the fashionpedia dataset regarding categories" "understand the attributes annotation process (can i train it separately, considering it as an instance segmentation problem with 294 attributes as classes)" These information can be found in our paper.

"where to find the Attribute-MASK RCNN model?": https://github.com/tensorflow/tpu/tree/master/models/official/detection/projects/fashionpedia

"Training Detectron2 on attributes, so that i can predict attributes from an image, independent of categories in image" You can certainly try that. One thing is that one mask may have more than one attribute, so your model should be able to predict multi-label instead of a single category like Mask-RCNN. And not all masks have attribute.

"To get clarity on, whether can i input any google image into fashionpedia API and expect the results to be classes + attributes from a single image? If yes, Please guide me through it!" Our predictor demo can do that. We are in the process of updating that file.

[bislara]

@KMnP Any update in the predictor demo file using the pre-trained model? It would be great if you can add it as soon as possible