ICNet-tensorflow icon indicating copy to clipboard operation
ICNet-tensorflow copied to clipboard

Training on cityscapes dataset

Open junweiy opened this issue 6 years ago • 3 comments

Hi,

I tried to run train.py to train the model from stretch on cityscapes dataset, and only commented the line that restores from pre-trained model and modified the path of dataset. My program managed to reach loss of 1 in the first 100 steps, however the loss continued to fluctuate around 1 until 5000 steps. I am wondering if there's anything that I should further modify, any ideas?

Thanks!

junweiy avatar Mar 13 '18 05:03 junweiy

I am also struggling to train from scratch on cityscapes

I am using the model with filter_scale=2 and I am using a flat learning rate of 0.01 using MomentumOptimizer. I am using proper data and have done plenty of testing to make sure that my evaluation is A OK.

I am wondering of the L2 loss is too heavily weighted. I know the paper mentions weight decay of 0.0001. After 10 hours of training, my L2 has come down from 1.0 to 0.5 and is still falling.

But has anyone actually successfully trained the cityscapes dataset from scratch using filter_scale=2? I would like to know how they have done it.

jinkos avatar Jun 22 '18 16:06 jinkos

Hi

Did you get success in training on cityscapes dataset? I'm not able to reproduce the result. The loss goes to around 0.5 but predictions are coming wrong(2-3% mIoU)

harora avatar Jan 21 '19 07:01 harora

I'm having the same problem... Loss went down but the mIoU is only 2%. Have you succeeded in training from scratch? Thanks!

Hi

Did you get success in training on cityscapes dataset? I'm not able to reproduce the result. The loss goes to around 0.5 but predictions are coming wrong(2-3% mIoU)

tzhong518 avatar Apr 29 '20 03:04 tzhong518