json label support for panoptic training

[griffintin]

Hi @yiyiliao , Thank you for your great work.

Will the json label of instance be available?

In the folder tree of http://www.cvlibs.net/datasets/kitti-360/documentation.php, instanceDict.json appears, but no in the downloaded data.
If I want to train for panoptic segmentation, how can I generate the training labels?

[yiyiliao]

Hi @griffintin , thanks for your question. You can find instanceDist.json here:

The 2D instance segmentation masks can be considered as labels for panoptic segmentation, where every pixel is labeled in the form of semanticId*1000 + instanceId.

[griffintin]

Hi @yiyiliao

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

And I found another problem. For every sequence, for part of RGB images in data_2d_raw, there is no corresponding instance .png in data_2d_semantics. If so, I think unnecessary ones better to be deleted. currenly, RGB images is about 83000 but instance is less than that.

[pwais]

@griffintin the "3d bounding box" data is in the data_3d_bboxes download and is not in JSON format. See this issue for some context that might be helpful: https://github.com/autonomousvision/kitti360Scripts/issues/6#issuecomment-729262235

[yiyiliao]

Hi @griffintin ,

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

Sorry I misunderstood your question. We do not provide instance.json per frame for panoptic segmentation. Good that you made them yourself!

And I found another problem. For every sequence, for part of RGB images in data_2d_raw, there is no corresponding instance .png in data_2d_semantics. If so, I think unnecessary ones better to be deleted. currenly, RGB images is about 83000 but instance is less than that.

This is true. We have some held-out instance labels that we plan to use for our benchmark while the corresponding RGB images have already been released.

[MichelleSchaaf]

Hi @yiyiliao

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

Hey, I am also trying to obtain an instance.json for training instance segmentation. However, when trying to collect binary masks for every single instance of an image, I have the problem that multiple objects in one image have the same label.

For example instance_segmentation 2013_05_28_drive_0000_sync/0000000253.png. There are multiple cars with the same instance label 101:

What am I missing? How can I get a binary mask of just one object and not the whole class?

Thank you in advance for any help!

[yiyiliao]

Hi @MichelleSchaaf, sorry for the late reply due to the summer break. Is there any chance that you did not load the instance mask in the format of uint16? Please refer to our documentation page:

Instance label in single-channel 16-bit PNG format. Each pixel value denotes the corresponding instanceID. Here, instanceID = semanticID*1000 + classInstanceID with classInstanceID denoting the instance ID within one class and classInstanceID = 0 for classes without instance label.