datumaro
datumaro copied to clipboard
convert to panoptic dataset with specified colormap
I would like to convert a dataset imported as coco detection or cvat to panoptic coco format by setting specified colormap for masks. I used the following code snippet for that. I could not find any information how to change colormap.
from datumaro.components.dataset import Dataset
# colormap = {"0": (0,0,0), "1": (73, 77, 174), "2": (220, 20, 60), "3": (70, 130, 180), "4": (64, 170, 64)} # 0 stands for background.
dataset = Dataset.import_from("annotations.json", "coco")
# dataset = Dataset.import_from("annotations.xml", "cvat")
dataset.transform('polygons_to_masks')
dataset.export('converted/', 'coco', save_images=True)
Hi. Our COCO implementation supports only instance-id-encoded masks (segments described in json and single-number indexed PNGs). If you want masks in color, consider exporting into the voc_segmentation
format.
Also, in the code snippet you provided, you'll probably need to add segmentation_mode='mask'
, if you want to load from COCO and save masks in COCO (by default, the code respects is_crowd
attribute, which will be kept untouched after polygons_to_masks
).
Thank you for quick reply. As far as I understood, I need to export in voc_segmentation format for panoptic labels. It is not clear for me how I can use exported labels in panoptic coco type. Default panoptic coco format includes color in segments_info to match png labels which is defined with categories. Also, I think your coco format conversion does not support the "is_thing" for panoptic segmentation.
Also, I would like to ask how you handle overlapping segments defined with polygons while converting from coco detection or segmentation to panoptic?
As far as I understood, I need to export in
voc_segmentation
format for panoptic labels. It is not clear for me how I can use exported labels in panoptic coco type.
I'll try to explain in more details. Apparently, there are at least 2 COCO panoptic formats out there - one with instance-id masks (no colors, described here, p.4) and another one with class masks (in color; probably, used for the stuff
task, which is not fully covered in Datumaro). Our COCO panoptic implementation supports only the first variant. Also, there is voc_segmentation
format, which should be close to the second variant with class masks in color, and in which you can control class colors.
format conversion does not support the "is_thing" for panoptic segmentation.
Yes, probably, you're right. It could be a good feature request.
how you handle overlapping segments defined with polygons while converting from coco detection or segmentation to panoptic?
The process is the following. Let's consider few terms:
- Spatial annotation - a polygon, bbox or mask
- They can have a
z_order
property set, which controls the "depth" or the "layer" of the annotation object. - Also, annotations can be joined into groups. Joined annotations have the same
group
attribute value - We call a group of annotations an "instance" (an annotated object)
- We collect all spatial annotations for an image and split them into groups (using the
group
attribute) - We look for the largest annotation (by the area) in each group and call it a
leader
. Thisleader
represents the instance (annotated object), which is being described by the group. The group annotations are merged into a single mask (if it is required by theis_crowd
attribute, or conversion mode, or if there is more than 1 annotation) or into a set of polygons (remember, COCO allows to have multiple polygons for a single object). The group inherits all the properties of itsleader
- includingz_order
, attributes, andlabel
. - If
crop-covered
is specified, we arrange obtained instances wrt theirz_order
(or don't change their order, if not specified), convert them into masks, paint them into a single image (which automatically crops covered parts), then convert back into source representation (polygons or a mask).
We provide several options to control behaviour:
-
--crop-covered
parameter allows to decide, whether we need to drop covered portions from underlying segments (false
by default) -
--segmentation-mode
(respectis_crowd
or keep input annotation type / force polygons / force masks)