UNET 3plus not learning on large datasets

[jrmathew1]

I am trying to use the UNET 3plus with deep supervision and cgm for nodule segmentation from lung ct scans. When I train the model on a 1000 image subset of the dataset the model learns and outputs meaningful segmentation masks.

However, when I train on a larger dataset of 2000 or more images the model does not learn and outputs a mask of all 0s always.

I have tried increasing and reducing the learning rate as well changing optimizers and loss functions but the issue remains. I also tried the att_unet_2d model and had the same issue.

I have attached my code: training_code.zip

[zhaxideles]

I also have the same problem, verifying that the loss is a constant, have you solved it？