Selim_sef densenet161 input shape wrong in docs

[Elhamahm]

Hello, I got the bug error “RuntimeError: Given groups=1, weight of size 96 4 7 7, expected input[4, 3, 320, 320] to have 4 channels, but got 3 channels instead” while training the model based on “selimsef_spacenet4_densenet161unet”.

My training dataset contains 640_640 RGB images with 3 channels, that have been cropped to 320_320, where the batch size is 4. My masked images are in the form of 1 channel images.

I would appreciate any help for solving this issue. Thanks

[nrweir]

Hi @Elhamahm,

Looks like there's a typo in our documentation indicating that the densenet model works for 3-channel imagery - it's designed for four channel inputs.

[Elhamahm]

Thanks for your comment. So we can convert our RGB three channels to 4 channels by adding extra columns (zeros or ones)... But if I should the same for masked images? Assuming that my original images are 3 channels and masked images are in 1 channel form, so how we should cope with this bug error? Thanks

[nrweir]

You're referring to the pixel masks that get compared to the prediction outputs, correct?

The densenet161 model that you selected is designed to take a three-channel mask. When it was originally trained, one channel indicated building footprints, another indicated their outlines, and the third denoted contact points where they were closely juxtaposed - like this:

You can try adding 2 more channels full of zeros to your mask if you'd like - depending on the loss function you select, though, results may be funky.

If you're not wedded to using the densenet161 architecture, i'd recommend you check out the XDXD model. That's a VGG16 encoder U-net that takes 3-channel inputs and generates a single channel output mask, and therefore should match the shape of your data.

[Elhamahm]

Thanks for your comprehensive comment... I have already used the XDXD model. Where I used "sol.vector.mask.foorptint()" for generating training masks (with 1 channel) applied in XDXD model. Could you please guide me can I use the same masking function (sol.vector.mask.foorptint) for generating mask images with 3 channels, that will be used as training masks for densenet161 model? Thanks

[nrweir]

Sure! Check out this tutorial.

[Elhamahm]

Hi @nrweir, In order to apply densenet161 model, I have generated three-channel mask images. Still, I am getting the following error while training the model based on three-channel mask images. So, I was wondering, the mask images must be in the form of three-channel or four-channel? Thanks

[Maxine3218]

Hi @nrweir, In order to apply densenet161 model, I have generated three-channel mask images. Still, I am getting the following error while training the model based on three-channel mask images. So, I was wondering, the mask images must be in the form of three-channel or four-channel? Thanks

Hi @Elhamahm , I have encountered similar dimension problems when trying to apply densenet161 model. Did you solve these? Any tips would be helpful. Thanks

[Elhamahm]

Hi,

In the case that you are sure about the number of images channels as it is 3, then I recommend you to drop these three parameters from augmentation (DropChannel, idx, axis,p) in config file in trainning_augmentation, validation_augmentation and Inference_augmentation. Hope it works.

Elham

On Mon, May 3, 2021 at 2:14 AM Maxine3218 @.***> wrote:

Hi @nrweir https://github.com/nrweir, In order to apply densenet161 model, I have generated three-channel mask images. Still, I am getting the following error while training the model based on three-channel mask images. So, I was wondering, the mask images must be in the form of three-channel or four-channel? Thanks [image: 77261868-5b6c4c00-6c57-11ea-9ba8-bb6854647b16] https://user-images.githubusercontent.com/7827229/79018836-09ba3180-7b32-11ea-9087-42be4b654abc.png

Hi @Elhamahm https://github.com/Elhamahm , I have encountered similar dimension problems when trying to apply densenet161 model. Did you solve these? Any tips would be helpful. Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CosmiQ/solaris/issues/341#issuecomment-831045442, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3W6HKMAEFFIG2RNREM4M3TLY5MFANCNFSM4LROVCDQ .

[Maxine3218]

Hi @Elhamahm

Thank you for your advices! With your help I managed to resolve this issue and trained the densenet161_unet.

Since I don't fully understand how selim(author of this code)'s approach to create final masks, I modified the yaml file and network file to train my densenet161_unet on 3-channel-RGB -> 1-channel mask training sets. But I got a weird training log and the results don't look so good. I'm not sure what caused this yet. I'm trying to train for more iterations, like 100 epochs? to see if the results could get better.

Did you manage to train your network on 3-channel-RGB -> 3-channel-masks? And if so, how's your results? Or would you mind share any ideas on what might induce this issue? I'll put my traing log visualization down below if that's useful.

Many thanks! Maxine3218

[Elhamahm]

In this link, it has been introduced that in the resnet, the input and masked images that will feed into the model must be three channels. https://solaris.readthedocs.io/en/latest/pretrained_models.html

For creating mask images, use this link... https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_masks_tutorial.html. according to that, I used

sol.vector.mask.footprint_mask (giving the input image and also geojson file as two input parameters)

For training the model, follow up this link https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_training_spacenet.html

Thanks,

Elham

On Wed, May 5, 2021 at 5:00 AM Maxine3218 @.***> wrote:

Hi @Elhamahm https://github.com/Elhamahm

Thank you for your advices! With your help I managed to resolve this issue and trained the densenet161_unet.

Since I don't fully understand how selim(author of this code)'s approach to create final masks, I modified the yaml file and network file to train my densenet161_unet on 3-channel-RGB -> 1-channel mask training sets. But I got a weird training log and the results don't look so good. I'm not sure what caused this yet. I'm trying to train for more iterations, like 100 epochs? to see if the results could get better.

Did you manage to train your network on 3-channel-RGB -> 3-channel-masks? And if so, how's your results? Or would you mind share any ideas on what might induce this issue? I'll put my traing log visualization down below if that's useful. [image: densenet161unet_loss] https://user-images.githubusercontent.com/40229356/117118123-fb852600-adc2-11eb-87a9-df621442b47b.png

Many thanks! Maxine3218

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CosmiQ/solaris/issues/341#issuecomment-832530947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3W6HOAKSMC5HFTTOJH5LDTMECLXANCNFSM4LROVCDQ .

[Elhamahm]

For training resnet/dense net, input images and masked images, both must be in three format channels. But, for training the model based on xdxd_spacenet, the input and mask images can be in the format of 1 channel.

For my case, I have trained models based on resnet and xdxd_spacenet, and both worked.

On Wed, May 5, 2021 at 5:21 AM Elham Ahmadi @.***> wrote:

In this link, it has been introduced that in the resnet, the input and masked images that will feed into the model must be three channels. https://solaris.readthedocs.io/en/latest/pretrained_models.html

For creating mask images, use this link... https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_masks_tutorial.html. according to that, I used

sol.vector.mask.footprint_mask (giving the input image and also geojson file as two input parameters)

For training the model, follow up this link https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_training_spacenet.html

Thanks,

Elham

On Wed, May 5, 2021 at 5:00 AM Maxine3218 @.***> wrote:

Hi @Elhamahm https://github.com/Elhamahm

Thank you for your advices! With your help I managed to resolve this issue and trained the densenet161_unet.

Since I don't fully understand how selim(author of this code)'s approach to create final masks, I modified the yaml file and network file to train my densenet161_unet on 3-channel-RGB -> 1-channel mask training sets. But I got a weird training log and the results don't look so good. I'm not sure what caused this yet. I'm trying to train for more iterations, like 100 epochs? to see if the results could get better.

Did you manage to train your network on 3-channel-RGB -> 3-channel-masks? And if so, how's your results? Or would you mind share any ideas on what might induce this issue? I'll put my traing log visualization down below if that's useful. [image: densenet161unet_loss] https://user-images.githubusercontent.com/40229356/117118123-fb852600-adc2-11eb-87a9-df621442b47b.png

Many thanks! Maxine3218

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CosmiQ/solaris/issues/341#issuecomment-832530947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3W6HOAKSMC5HFTTOJH5LDTMECLXANCNFSM4LROVCDQ .

[Maxine3218]

Hi Elham

Thanks for your quick reply!

The original network outputs three masks, so I modified the last layer in the Encoder-Decoder network to output only one mask, just like xd-xd’s vgg16_unet model.

I’ve trained my modified densenet161-unet model, but the results was far worse than vgg16_unet trained with the same dataset. I’m quite confused by what might cause this.

I’ve successfully create 3-channel masks with that function, but I’m having troubles understanding how to create the final building masks with the footprint+boundary +contact masks, so I decided not to train 3-channel-outputs models yet. I’ve read selim’s solutions and he seems to have used the LightGBM+watershed approach to create final masks. I’m still trying to understand how to do these.

Thank you so much for your patience. You’ve been of great great help to me.

Maxine

On May 5, 2021, at 5:27 PM, Elhamahm @.***> wrote:

 For training resnet/dense net, input images and masked images, both must be in three format channels. But, for training the model based on xdxd_spacenet, the input and mask images can be in the format of 1 channel.

For my case, I have trained models based on resnet and xdxd_spacenet, and both worked.

On Wed, May 5, 2021 at 5:21 AM Elham Ahmadi @.***> wrote:

In this link, it has been introduced that in the resnet, the input and masked images that will feed into the model must be three channels. https://solaris.readthedocs.io/en/latest/pretrained_models.html

For creating mask images, use this link... https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_masks_tutorial.html. according to that, I used

sol.vector.mask.footprint_mask (giving the input image and also geojson file as two input parameters)

For training the model, follow up this link https://solaris.readthedocs.io/en/latest/tutorials/notebooks/api_training_spacenet.html

Thanks,

Elham

On Wed, May 5, 2021 at 5:00 AM Maxine3218 @.***> wrote:

Hi @Elhamahm https://github.com/Elhamahm

Thank you for your advices! With your help I managed to resolve this issue and trained the densenet161_unet.

Since I don't fully understand how selim(author of this code)'s approach to create final masks, I modified the yaml file and network file to train my densenet161_unet on 3-channel-RGB -> 1-channel mask training sets. But I got a weird training log and the results don't look so good. I'm not sure what caused this yet. I'm trying to train for more iterations, like 100 epochs? to see if the results could get better.

Did you manage to train your network on 3-channel-RGB -> 3-channel-masks? And if so, how's your results? Or would you mind share any ideas on what might induce this issue? I'll put my traing log visualization down below if that's useful. [image: densenet161unet_loss] https://user-images.githubusercontent.com/40229356/117118123-fb852600-adc2-11eb-87a9-df621442b47b.png

Many thanks! Maxine3218

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CosmiQ/solaris/issues/341#issuecomment-832530947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3W6HOAKSMC5HFTTOJH5LDTMECLXANCNFSM4LROVCDQ .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.