segmentation_models
segmentation_models copied to clipboard
qubvel
Questions for CategoricalCELoss
Hi guys,
I would like to ask you some questions. I am trying to train a UNet model with a ResNet encoder (exactly like the one that you provide in the jupyter notebook example) to semantically segment my images. My images have 2 or more classes (I want to use the same model more just once -- solve different problems).
I found out that in your Dataset class, you insert a background value as a different class. I don't need that, so I commented that out. Will this affect any connected functionality?
Furthermore, I would to use categorical_crossentropy_loss. Should I try to one-hot-encode my labels? Is there something that I need to do to make your example work properly with this loss function? How can I call this function using your sm.losses?
Thank you for your help. I think that they are relatively easy questions and that you will answer them right away. Have a nice day!
I don't need that, so I commented that out. Will this affect any connected functionality?
In datasets where there are some pixels left unlabeled, adding a background class serves as a sort of catch-all for objects in the image that you don't care about. For example, all objects within the scene that were not the three classes that were specified all get grouped into a single class. If every single pixel in your dataset is labelled, then having a background class isn't necessary.
Should I try to one-hot-encode my labels?
Yes. You need to one-hot-encode them to work with Keras regardless of the loss function you use. To make things easy, you should have your masks in the format of (H, W, C), where C is the number of channels and is equal to one, and where each index within the channel is some value from 0 to N, the number of classes you are concerned about. Once in this format, you can use keras.utils.to_categorical to create a one-hot-encoded mask for training.
To use the loss function, you can do something like:
my_loss_function = sm.losses.Categorical_Cross_Entropy(class_weights = my_class_weights)
...
model.compile(loss = my_loss_function, optimization = keras.optimizers.Adam(), metrics = ['accuracy'])
Hope that helps!
Let's assume that I have three classes. Background, class A and class B. Basically, I need to identify every pixel that belongs to class A and class B and the rest will be classified as background. What kind of weights should I use for class A and B? Is this a custom implementation of the Keras cross_entropy_loss with weights?
Furthermore, what kind of metrics and callbacks do you suggest that I should use for this case? How will the choice of metrics affect the training of the model? Do you use Learning Reduce on Plateu callback?
Also, since I care only about particular classes, shouldn't I only care about the accuracy that I achieve on those classes instead of the global accuracy? Is there a way to achieve that?
Thanks for your advice.
Weights as in class weights: you could try to calculate the frequency of each class based on the amount of pixels for each class per image within the training dataset and then pass that ratio to the loss function you're using. This is useful in situations where the classes are imbalanced. So if the vast majority of pixels in each image belong to background, then it will have a lower weight to it than the other classes.
You can use sm.losses which is custom and it allows you to pass in a list representing the class weights. There are also other useful losses like focal and dice.
Metrics will only tell you how good your model is doing during training and validation, it doesn't necessarily affect the results of the model itself. If you change the loss function though, the model itself will be altered. But if you're working on a multi-class classification, any of the categorical loss functions are your best bet.
Callbacks are useful, I used both Model Checkpoint and ReduceLR, it's really up to you. Try training and get a feel for when overfitting occurs and then set up ReduceLR based on that. Try different optimizers as well. SGD is standard but I find Adam converges more quickly.
The metrics provided by this repo will give you the scores globally (averaged across each class), but you can find or create your own metrics that only look at each class or simply just zero-out the pixels for the classes you don't care about.
Yes. You need to one-hot-encode them to work with Keras regardless of the loss function you use. To make things easy, you should have your masks in the format of (H, W, C), where C is the number of channels and is equal to one, and where each index within the channel is some value from 0 to N, the number of classes you are concerned about. Once in this format, you can use
keras.utils.to_categoricalto create a one-hot-encoded mask for training.
Hi again. I used your segmentation suite quite a lot and it's working properly. Do I really need to do a one-hot-encoding? It seems that your example contains a functionality that already converts the input masks to a H,W,C format and then training runs properly.
Hi again. I used your segmentation suite quite a lot and it's working properly. Do I really need to do a one-hot-encoding? It seems that your example contains a functionality that already converts the input masks to a H,W,C format and then training runs properly.
If your masks are already in the format H, W, C, where C is equal to the number of classes you're training, and each one is composed of 0's and 1's signifying where that class is present within the channel, then you don't need to one-hot-encode them, because they're already in the correct format.
Depending on how you create masks originally, to_categorical() can be used to put them in one-hot-encoded form if they aren't already.
If what you're doing is working, then they are already in the correct format.
