Difference in the mean IoU on test dataset

[Sreeni1204]

Hello,

Thanks for the good work.

I have a few questions to ask.

1. Is the step "1-15" under task "15-5" and step "1-15" under task "15-5s" the same? If yes, why is there a difference in the performance or mean IoU values in the paper?

2. I performed the FT and MiB experiments, and the mean IoU is differing too much from the one in the paper. The difference between the command given by you and the one I am using is that I have a single CUDA device and reduced the batch size to 12.

Thanks and regards, Sreeni...

[fcdl94]

Hello @Sreeni1204,

1. Yes, the model is the same. However, the performance is reported after the last incremental step is done. Thus, doing one incremental step or doing multiple give indeed different results.
2. Well, if you reduce the batch size, try to decrease also the learning rate (our empirical rule is to divide the LR by the same rate - in your case, halve it). Remember also that experiments in the 15-1 scenario used a loss_kd value of 100, while for all the other settings we used 10.

Sorry for the late answer.