Pretrained models

[EliasVansteenkiste]

Is it possible to make the weights of the trained models available?

[bodokaiser]

@saeedizadi maybe has some?

[saeedizadi]

I've modified the models by @bodokaiser for a binary segmentation problems on medical images (using nn.Sigmoid + BCELoss) and hence my pretrained models may not come helpful in general. However, the training is very straight forward on other datasets. Let me know if you help.

[duylp]

Hi @saeedizadi , can you share what you changed for the binary segmentation? I have set the number of classes = 2 everywhere and used a Sigmoid + BCELoss. However, I got some weird results. How did you modify the output of the net with shape (2xHxW) to match to the mask's shape (1xHxW)? Thanks!

[saeedizadi]

Hi @ZweeLe Yes, I wanna share the modifications but I'm a bit out of time these days. You must change the num_classes to 1 instead of 2. Then, simply change the last layer activation function to nn.Sigmoid() and employ nn.BCELoss() in your code.

Let me know if you need more information. Saeed

[duylp]

@saeedizadi Thanks! I have tried to set num_classes to 1 and got zeros everywhere in the output mask. It turned out that I forgot to apply the Sigmoid to the output. Now, I got very good results. Thank you a lot!

[andrewssobral]

Hi @saeedizadi , I am trying to do the same as @ZweeLe (binary segmentation), but I'm lost. When you talk about:

You must change the num_classes to 1 instead of 2. Then, simply change the last layer activation function to nn.Sigmoid() and employ nn.BCELoss() in your code.

It means:

#NUM_CLASSES = 22
NUM_CLASSES = 1

and

class CrossEntropyLoss2d(nn.Module):

    def __init__(self, weight=None):
        super().__init__()

        #self.loss = nn.NLLLoss2d(weight)
        self.loss = nn.BCELoss(weight)

    def forward(self, outputs, targets):
        #return self.loss(F.log_softmax(outputs), targets)
        m = nn.Sigmoid()
        return self.loss(m(outputs), targets)

?

[andrewssobral]

Hi @ZweeLe , Can you share your modifications for binary segmentation? Thanks

[bodokaiser]

I think you are right that binary segmentation only needs 1 output class / channel with 0 corresponding to first and 1 corresponding to second class.

So something like:

model = Net(1)

criterion = nn.BCEWithLogitsLoss()

outputs = model(inputs)

loss = criterion(outputs, targets)

should work. Further check outputs and targets to check if they are compatible with what BCEWithLogitsLoss expects.

[duylp]

Hi @andrewssobral , you just need to follow the above modifications of @bodokaiser. For calculating the loss, the outputs and targets must be 1D arrays: loss = criterion(outputs.view(-1), targets.view(-1)) To use the nn.BCEWithLogitsLoss() you have to build Pytorch from source. For me I used nn.Sigmoid() and nn.BCELoss() but they are less numerically stable. For the evaluating code, you will need another post processing code:

outputs = model(inputs)
mask = sigmoid(outputs) * 255
mask[mask < thres] = 0
mask[mask >= thres] = 255

"thres" is the threshold pixel value you set to make a binary mask. You can find out the best value with validation (mine is 120). If you use the default transform then you don't need to multiply with 255.

[andrewssobral]

Thank you @ZweeLe ! When you say:

To use the nn.BCEWithLogitsLoss() you have to build Pytorch from source. For me I used nn.Sigmoid() and nn.BCELoss() but they are less numerically stable.

It means?

class CrossEntropyLoss2d(nn.Module):

    def __init__(self, weight=None):
        super().__init__()

        #self.loss = nn.NLLLoss2d(weight)
        #self.loss = nn.BCEWithLogitsLoss(weight)
        self.loss = nn.BCELoss(weight)

    def forward(self, outputs, targets):
        #return self.loss(F.log_softmax(outputs), targets)
        #return self.loss(outputs, targets)
        return self.loss(nn.Sigmoid(outputs), targets)

[duylp]

@andrewssobral Yes, they are similar. However, the BCELoss only accepts 1D tensors so you have to reshape your tensors or you can define a new BCELoss2d function.

[andrewssobral]

Thank you @ZweeLe

I did your tips, and now the algorithm is running, but the loss becomes negative, please see:

loss: 1.2735891342163086 (epoch: 1, step: 0)
save: segnet-001-0000.pth (epoch: 1, step: 0)
loss: -1.1411563158035278 (epoch: 1, step: 1)
loss: -3.6113040447235107 (epoch: 1, step: 2)
loss: -4.822791635990143 (epoch: 1, step: 3)
loss: -6.884974336624145 (epoch: 1, step: 4)
loss: -7.026113708813985 (epoch: 1, step: 5)
loss: -8.921686070305961 (epoch: 1, step: 6)
loss: -10.695589691400528 (epoch: 1, step: 7)
loss: -11.778627316157023 (epoch: 1, step: 8)
loss: -13.278458523750306 (epoch: 1, step: 9)
loss: -14.816673040390015 (epoch: 1, step: 10)
loss: -16.244035343329113 (epoch: 1, step: 11)
loss: -17.84725154363192 (epoch: 1, step: 12)
loss: -19.353035739489965 (epoch: 1, step: 13)
loss: -21.03988642692566 (epoch: 1, step: 14)
loss: -22.63021345436573 (epoch: 1, step: 15)
loss: -23.58169169986949 (epoch: 1, step: 16)
loss: -25.269151012102764 (epoch: 1, step: 17)
loss: -26.97707804880644 (epoch: 1, step: 18)
loss: -28.725287783145905 (epoch: 1, step: 19)
loss: -29.03264051391965 (epoch: 1, step: 20)
loss: -30.018881505185906 (epoch: 1, step: 21)
loss: -31.15997344514598 (epoch: 1, step: 22)
loss: -32.05112152298292 (epoch: 1, step: 23)
loss: -33.00953503608704 (epoch: 1, step: 24)
loss: -35.36375263104072 (epoch: 1, step: 25)
loss: -36.99868559837341 (epoch: 1, step: 26)
loss: -39.33454824345453 (epoch: 1, step: 27)
loss: -41.40287765141191 (epoch: 1, step: 28)

this is normal?

[duylp]

@andrewssobral That is weird, the loss must be positive and decrease over time. You should check your outputs of the model and the targeted masks before applying BCELoss. All values should be in range of 0~1. Did you apply the transformation to the labels?

[andrewssobral]

Hi @ZweeLe , I solved this issue by changing this:

input_transform = Compose([
    CenterCrop(256),
    ToTensor(),
    Normalize([.485, .456, .406], [.229, .224, .225]),
])
target_transform = Compose([
    CenterCrop(256),
    ToLabel(),
    Relabel(255, 21),
])

to this:

input_transform = Compose([
    Scale((64,64)),
    ToTensor(),
])
target_transform = Compose([
    Scale((64,64)),
    ToTensor(),
])

Now, the algorithm is learning well, the loss is decreasing and the preliminary results are very good! Thank you! and thanks to @bodokaiser for your advices ;-)

[duylp]

@andrewssobral Yes. The ToLabel() and Relabel() are not necessary for binary segmentation. Can you share your training time? For me, it took nearly 5 secs for an iteration and 0.5s to evaluate an image, which is a bit slow since I have a large dataset.

[andrewssobral]

@ZweeLe as I have only 126 images and I'm using Amazon EC2 with p2x.large instance (NVIDIA K80 with 12Gb of RAM), the training time is very fast (less than 5 minutes). In my laptop with only CPU, it is much more slow, I stopped the training berfore 10 epochs. I think the training will take around (or more than) 30min for the whole dataset.

[woaichipinngguo]

Can I directly use nn.CrossEntropyLoss() for binary segmentation? Need I set num_class=1 ? And I want use weighed loss. @andrewssobral @duylp @saeedizadi @bodokaiser

[saeedizadi]

You would suggest to use Sigmoid() + BinaryCrossEntropy for Binary Segmentation.

[woaichipinngguo]

@saeedizadi I don't understand how to set the parameter weight for : class torch.nn.BCELoss(weight=None, size_average=None, reduce=None, reduction='elementwise_mean')

[saeedizadi]

@woaichipinngguo You should see the definitions for the parameters. But, for the initial try, just try to use the default values, and see the results, then, you can customise them toward your taste

[andrewssobral]

Hi @woaichipinngguo , you can find here an example of training binary segmentation on pytorch here: https://github.com/andrewssobral/deep-learning-pytorch/blob/master/segmentation/train_binseg.py this code was based on bodokaiser/piwise repository. However, in the code I don't make use of weights, and some improvements (e.g. model selection, prediction, etc) are needed, I still working to improve it.

[woaichipinngguo]

when I set num_class=1 and use NLLLoss() loss, I will encounter an error: RuntimeError: Assertion cur_target >= 0 && cur_target < n_classes' failed. at /opt/conda/conda-bld/pytorch_1525812548180/work/aten/src/THNN/generic/SpatialClassNLLCriterion.c:111` @andrewssobral

class UNet(nn.Module):
    def __init__(self, n_channels=3, n_classes=1):
        super(UNet, self).__init__()
        self.down=resnet18()
        self.inc = inconv(n_channels, 16)
        self.down1 = down(64, 128)
        self.down2 = down(128, 256)
        self.down3 = down(256, 512)
        self.down4 = down(512, 512)'''
        self.up1 = up(96, 64,64,drop=0.2)
        self.up2 = up(80, 32,64,drop=0.2)
        self.up3 = up(48, 16,32,drop=0.1)
        self.up4 = up(32, 16,16,drop=0.1)
        self.outc = outconv(16, n_classes)
        self.reset_params()

        criterion=nn.NLLLoss()
        for ii,(data,mask,_)in enumerate(trainloader):
            #训练模型
            data,mask=data.to(device),mask.to(device)
            mask=mask.long()   
            optimizer.zero_grad()
            out=myNet(data)#numclass=1
            out = F.log_softmax(out, dim=1)
            loss=criterion(out,mask)

[jizongFox]

@woaichipinngguo You may have a label index overpassing the limit. Cross-entropy needs at least 2 classes.