How to train a custom dataset with custom number of classes

[RakhithaRR]

Expected results

I want to train a custom database with 3 classes on the model 'e2e_mask_rcnn_R-50-FPN_1x'. I have set up the database according to the COCO format and I added the necessary codes in dataset_catalog.py and train_net_step.py.

I ran the command python tools/train_net_step.py --dataset customdb --cfg configs/baselines/e2e_mask_rcnn_R-50-FPN_1x.yaml --use_tfboard --bs 1 --nw 4.

I get several runtime warnings in boxes.py saying "invalid value encountered in maximum/minimum" and an error in lib/roi_data/mask_rcnn.py saying "IndexError: index 0 is out of bounds for axis 0 with size 0"

Any help with this would be appreciated.

System information

• Operating system: Ubuntu 16.04 LTS
• CUDA version: 9
• cuDNN version: 7
• GPU models (for all devices if they are not all the same): GTX 850M 4GB x1
• python version: 3.6
• pytorch version: 0.4

Also, I used ImageNet pretrained weights R-50.pkl for ResNets.

[matchumen]

@RakhithaRR Have you made any progress? I got stucked at the same problem.