MONAILabel
MONAILabel copied to clipboard
0.5 threshold used in multi-label deepedit
Describe the bug In the codebase (here, here and here) the threshold of 0.5 is used disregarding the number of labels.
Expected behavior The threshold should be 1/(len(labels)-1). Such that for a 3-class problem (besides background) the threshold is 0.33.
At the moment we are asking the models to be extra sure of the labels before considering them right. This might be useful but is not explicitly implemented and is worse and worse the more classes there are.
https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/deepedit/transforms.py#L770 and https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/deepedit/transforms.py#L770 are used during the training and the comparison is against the label (truth values) which are normally expected to be 1 or above. so, label > 0.5 is not an issue.
however, https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/deepedit/transforms.py#L461 this one makes sense. currently pred (eval output) is always checked against 0.5 threshold value and it can have different values for differ labels (to give different priority for each of label type).
There are 2 options to unblock if you are stuck here..
- Extend the FindDiscrepancyRegionsDeepEditd and override the
def _apply(self, label, sid)
method to fix the behavior accordingly. - Fix monai app transform directly and support for label-wise threshold values (1/(len(labels)-1 is one kind and let user chose and pass the threshold values of their choice instead of this one kind)
Fix monai app transform directly and support for label-wise threshold values (1/(len(labels)-1 is one kind and let user chose and pass the threshold values of their choice instead of this one kind)
I will work on a PR for this this week.