Does your code contain the mask mAP computation?

[wuleibupt]

[souryuu]

The one in fix_testing branch is included with evaluation API from coco. However, I have change many things on the branch. I think you can not directly use it to evaluate your trained network. You may need to retrain the network.

[souryuu]

With current version, I added the evaluation part using API from MSCOCO for both bounding box and mask. You can evaluate the network by

python train/test.py
python pycocoEval.py

After training up to 1M+ iteration, I got mAP(mask IoU=0.5) on training data = 0.47 and on testing data = 0.36 (comparing to original paper Resnet 50 mAP of 0.512). Please note that your previously trained networks and created tfrecords may not work with this version because I changed a lot of network settings.

I found 2 problems but I will not have time to fix it. 1). I tried to set is_training in Resnet and header to False during training but the performance dropped to ~0.20. If is_training is set to true, I got the above performance . I think this has something to do with batch normalization layers. Probably batch_size =1 is the problem. However, it seems that tensorflow does not support for batching images with dynamic shape. 2). When training with too high/low aspect ratio images, the convolutional features in pyramid become too large and GPU goes OOM. In order to train the current version, I removed too high/low aspect ratio images from the tfrecords.