Loss stuck in DenseVnet training

[KathrynSch]

Hi, I am trying to train the dense_vnet network on my own dataset (30 abdominal CT scans fro liver segmentation). The loss (dice_hinge.dice) got stuck at 0.24 after 1000 iteration and kept oscillating around that mean value for another 1000 iterations before I interrupted it. I then tried to overfit on a single sample (as a sanity check of the network). Same thing happened: stuck around 0.23 after 40 iterations (that I let go on for 600 iterations...). Other details: using resize sampler (to 144144144 volumes), no augmentation, adam optimizer as it is was set in the default config file (pretrained densevnet). Has this happened to anyone ? Any idea what might be going wrong here ? Thanks a lot for your help

[ericspod]

The discussion in #208 may be relevant to your case. It is hard to say without seeing your data set, but the issues of window size, batch size, initialization weights, or relative size of labels may be relevant to you.