Blocky segmentation masks

[aleksmirosh]

Hello, thank you for your incredible work. This model shows great IoU on my data, much better than all I tested before. But even after finetune masks look blocky. Like here, like small squares:

Do you have some suggestions please how I can deal with it? I finetuned with sen1floods11 config and checkpoint.

[aleksmirosh]

any suggestions?

[HamedAlemo]

@aleksmirosh we had another user trying this and having similar pattern in their prediction. The issue for them turned out to be that their input data was between [0, 1] but you need to multiply them by 10000 to be integers. cc @paolofraccaro

[aleksmirosh]

thank you @HamedAlemo for your response, I much appreciate it. I tried the model on satellite but not Sentinal data, it is already between 0 and 60000. I fond that this block is 16x16 size. Is this the problem with the encoder/decoder? Should I finetune it? I much appreciate your help with the issue

[HamedAlemo]

I think @thesujitroy or @paolofraccaro should be able to help with this.

[CarlosGomes98]

Hi @aleksmirosh . This is somewhat confusing to me, as the predictions should be either 0 or 1 for each pixel, so completely dark or completely white, So I suspect at least some of this is due to distortion being introduced by compression in your image (are you saving it as JPEG perhaps?)

Its possible the output of the image is also being affected by the 16x16 patch size though, as you observed. You could try using a smaller patch size on your data, although it may take some more training epochs.

[aleksmirosh]

Hi @CarlosGomes98 thank you for the response. I much appreciate your time. My predictions are probabilities, I tested the original finetune script on the provided sentinel floods, and predictions were probabilities between 0 and 1 too. I attached mask I got after original finetune process on flood data.

For to my custom finetune the input is TIFF and I save it as TIFF. I tried different constants and normalizations, but the results are the same.

I will try to experiment with patch size. Does it make sense to try a bigger patch size? In this case, I must use batch size = 1, just memory limit.