ttach icon indicating copy to clipboard operation
ttach copied to clipboard

merge_type = "tsharpen" cause "nan"

Open JiaquanYe opened this issue 6 years ago • 4 comments

I use the example like: model = tta.SegmentationTTAWrapper(model, tta.aliases.d4_transform(), merge_mode="tsharpen")

when I use this model to predict, I found the output of model has value "nan"...

I look up the source code of this project, I have found when tsharpen model will do : x = x**0.5

Is it the negative value in the tensors pass throught this operation will cause "nan"?

JiaquanYe avatar Dec 02 '19 08:12 JiaquanYe

HI, I am not sure if thats right, did you solve the problem?

qubvel avatar Dec 11 '19 11:12 qubvel

I change my code use tta.transforms instead of TTAWrapper. Then I can do normalization to the tensors(make it to [0,1]), and that could avoid the negative value in tensors. So x=x**0.5 will not bring "nan".

I think many models will do normalization before sending images to it like this: torchvision.transform.Normalize(mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225]). In this case, the output tensors of model usually get negative value. If use tta.SegmentationTTAWrapper in merge_mode="tsharpen" directly will cause nan.

So I think we should scale the output tensors to [0, 1] to avoid this problem.

JiaquanYe avatar Dec 11 '19 12:12 JiaquanYe

The same story with gmean. In my case ResNet has FC defined by nn.Linear which outputs both positive and negative values. And ClassificationTTAWrapper in merge_mode='gmean' draws some nans.

pgsrv avatar Feb 26 '20 23:02 pgsrv

Use sigmoid/softmax activation for model output to avoid nan values

qubvel avatar Feb 27 '20 06:02 qubvel