The mIoU on the cityscapes dataset is low

[fyn111]

I use local_configs/segformer/B5/segformer.b5.1024x1024.city.160k.py to train a model, but the mIoU is low even after 63950 iterations: +---------------+-------+-------+ | Class | IoU | Acc | +---------------+-------+-------+ | road | 0.81 | 39.61 | | sidewalk | 90.83 | 96.99 | | building | 68.91 | 94.91 | | wall | 22.03 | 24.51 | | fence | 13.55 | 51.07 | | pole | 11.99 | 56.55 | | traffic light | 11.77 | 20.65 | | traffic sign | 89.88 | 97.59 | | vegetation | 55.71 | 82.64 | | terrain | 21.43 | 85.35 | | sky | 8.41 | 9.78 | | person | 57.04 | 97.61 | | rider | 27.36 | 68.83 | | car | 22.67 | 77.23 | | truck | 19.68 | 47.09 | | bus | 13.06 | 65.01 | | train | 0.0 | nan | | motorcycle | 31.27 | 75.63 | | bicycle | 0.46 | 0.47 | +---------------+-------+-------+ 2021-09-20 08:44:46,087 - mmseg - INFO - Summary: 2021-09-20 08:44:46,088 - mmseg - INFO - +--------+-------+-------+-------+ | Scope | mIoU | mAcc | aAcc | +--------+-------+-------+-------+ | global | 29.83 | 60.64 | 90.59 | +--------+-------+-------+-------+

[Inch-Z]

I have the same question on the ADE20K, about 0.6% lower than the paper report

[Mordokkai]

@fyn111 I had also a low score on ADE20K and it was due to the low batch size I was using. I'm not used to multigpu training so I didn't realize that the batch size was computed by: number_gpus*samples_per_gpu. By default in their code samples_per_gpu=2 which means that they probably used 8 gpus since in their paper they said they used a batch_size of 16 for ADE20K. I was using only one gpu so my batch_size was significantly lower than theirs. By increasing samples_per_gpu to 6 I already got much better results. I have no idea if that's your problem, but maybe you should try to increase samples_per_gpu

[cooperleong00]

The paper says they pre-trained the model on Image-net, have you tried pre-training?

[cuiziteng]

It seem you haven't load pre-train model or just train on 1 GPUs and didn't adjust learning rate. Here is my results:

+---------------+-------+-------+ | Class | IoU | Acc | +---------------+-------+-------+ | road | 98.33 | 99.02 | | sidewalk | 86.38 | 93.75 | | building | 93.55 | 97.01 | | wall | 67.6 | 74.32 | | fence | 64.07 | 71.14 | | pole | 69.53 | 79.94 | | traffic light | 75.17 | 84.61 | | traffic sign | 81.84 | 88.69 | | vegetation | 93.12 | 96.99 | | terrain | 66.42 | 74.3 | | sky | 95.3 | 98.57 | | person | 84.98 | 92.43 | | rider | 68.77 | 82.45 | | car | 95.89 | 98.07 | | truck | 87.63 | 90.3 | | bus | 90.39 | 95.18 | | train | 82.65 | 87.15 | | motorcycle | 73.5 | 83.6 | | bicycle | 80.45 | 90.49 | +---------------+-------+-------+ 2022-04-09 14:31:53,589 - mmseg - INFO - Summary: 2022-04-09 14:31:53,589 - mmseg - INFO - +-------+-------+-------+ | aAcc | mIoU | mAcc | +-------+-------+-------+ | 96.62 | 81.87 | 88.32 | +-------+-------+-------+

[xibi777]

I train b0_cityscapes on 1 GPU, and I only change the samples_per_gpu into 8. It performs not well, and it seems that the model overfit after 60000 iterations. Should I also change my learning rate? Thanks!!!