ESANet compute_class_weights issue for 894 labels

Hello, thank you so much for making the code publicly available. It is very helpful and easy to read and follow. I'm facing an issue with compute_class_weights function for 894 labels. For 13 and 40 labels it is working fine, but for 894 labels it show class weighting contains NaN. Not sure what might be the reason. Is it because of the classMapping.mat file? can you explain this file's role?

Jun 24 '21 03:06 gnanukoth

Maybe there are some categories which actually do not appear in the ground truth segmentation. This would result in zeros in the variable n_image_pixels_with_class and lead to NaN in division

Jun 29 '21 08:06 mona0809

Thanks for getting back. I need a clarification regarding the depth image format for training and inference. For testing on custom procured RGB and depth images, what format should they be saved in? I tried saving the depth in mm as uint16 format. But the prediction results are not great, even after depth filling using colorisation technique. Is this approach right? Do you have any pointers for real time inference? May I know what depth map processing technique you followed in your robot system deployment experiment?

Jul 08 '21 07:07 gnanukoth

It depends on the dataset:

nyuv2: in mm, uint16 sundrgbd: in 1/10 mm, uint16 cityscapes: in m, float16

For application, we used sunrgbd model with images of a Kinect2. We did not use any depth inpainting methods. Just pay attention to use the parameter --raw_depth so that invalid depth values are set back to 0 after normalizing.

Aug 18 '21 14:08 mona0809