Questions about training time and inpainting real images

[xhh232018]

Hi, according to what you mentioned in #6 . I have to set the pixel value of broken parts to 0, which is the mask area like this: How can I input them into the trained completion model based on my own dataset and get the complete result? Also, I found that chainer will save the model file（.npz) every 10 epochs so that the file size is quite large. The training process has to be terminated. Therefore, I want to know how to solve this problem. Last, could you give me some suggestions of a rational training time? Thanks for your help and looking forward to your reply.

[SeitaroShinagawa]

Thanks for your interest.

Answer to the first question
As shown in https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/common/net.py#L56,
original images and masks are element-wise multiplied to get broken images "x*mask."

You can replace this "x*mask" with your broken input. To do this, an easy way is to set x by your broken image and mask by ones array.

Of course, you can rewrite the line without mask like this,

    #current
    def __call__(self, x, mask):
        self.m.W.data = self.xp.array(self.maskW) #mask windows are set by 1
        h = self.c(x*mask) #(B,C,H,W)

    #modified
    def __call__(self, x, mask=None): #x denotes broken image array
        self.m.W.data = self.xp.array(self.maskW) #mask windows are set by 1
        if mask==None:
            h = self.c(x) #(B,C,H,W)
        else:
            h = self.c(x*mask) #(B,C,H,W)

Answer to the second question
you can change save timing by editing https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/train.py#L120

Answer to the last question
Training time depends on your setting.
As far as my experience, it takes

• around 4 days with GTX1080, batch_size=4, image_size=256, 300,000 iteration.
• around 5 days with TeslaP100, batch_size=8, image_size=512, 500,000 iteration.
  Note that it is a setting without fine-tuning stage. I think the former setting seems not enouph to get good performance.

I hope it helps you.
Best,

[xhh232018]

Hi, Seitaro. Thanks for your help. Forgive my carelessness, I should have solved the second question. The size of my own dataset is 256 and I have trained them for 2 days. However, the results seem to be imperfect. As those 2 pictures: We can see that there are blue regions in the 4 corners of each image, which corresponds to your result in README.md. You said the former setting is not fine-tuning. I wonder whether I can solve this problem by fine tuning. Also, you said this implement is different from the original one. Is this problem related to the structure of the net? The result from the paper is quite beautiful so I believe you want to solve this problem. Thanks for your help and looking forward to your reply.

[SeitaroShinagawa]

It seems because the external cropping loss of the cropping frame is ignored in my implementation.
As discussed in #12, I think you can avoid this problem if you can set the input size of pre-trained model by 224x224.