Can I train with patches?

[PerdonLiu]

What a wonderful and impressive work!

But I was a little confused about the detail. The following is my understanding, could you please point out whether it is right or not?

Bagnet is nothing different from resnet both in training and testing procedure except that some of the 3*3 convs are replaced with 1*1 convs for smaller receptive field. In your code, you feed a large image(224*224) to bagnet33 in training. Actually it is equivalent that we feed the same image in patches to bagnet33 which are generated by function generate_heatmap_pytorch (patchsize=33, stride=8) and average the logits of each patch.(is it right?)

But when I tried feeding the same image in patches and averaging the patches result in training, it seemed something was wrong and its prediction result was bad. I was wondering if it was something I didn't quite understood?

Thanks in advance and I can't help saying that this paper is extremely useful!

[wielandbrendel]

Your understanding of the paper and the model is correct. Are you making sure that you use the correct preprocessing on the individual patches?

[PerdonLiu]

Thanks for your reply! The following is what I did in detail:

I first preprocessed the large images(224*224) as usual and then generated small patches and fed all individual patches to bagnet33 like generate_heatmap_pytorch to get logits of size (Batchsize, 26*26, num_classes). I then averaged the logits over the second dimension(26*26) to get the final results of size (Batchsize, num_classes).

But the results followed above seems not so good in the training process. Is it something that I have misunderstood or missed? Looking forward to your kind reply. Thanks!

[wielandbrendel]

@PerdonLiu Your procedure looks fine in general so there must be some devil in the details (the procedure does work: if you choose the correct stride you get identical results for the two procedures). What patch size and what stride are you using?

[PerdonLiu]

Thanks to your reply!

In the generate_heatmap_pytorch method, I use patchsize=33 and stride=8 with padding just as what you did. Now the situation is :

In the testing process, the generating patches procedure works normally. But in the training process, it does not work if I turn the model.train() on. When I just turn the model.eval() on, it works even in training. I guessed It seems to be something wrong, right? But I have no ideas yet.

Can you reproduce the results in these two procedure? Looking forward to your patient reply! Thanks!

[Fatima-Ito]

Thanks for your reply! The following is what I did in detail:

I first preprocessed the large images(224224) as usual and then generated small patches and fed all individual patches to bagnet33 like generate_heatmap_pytorch to get logits of size (Batchsize, 2626, num_classes). I then averaged the logits over the second dimension(26*26) to get the final results of size (Batchsize, num_classes).

But the results followed above seems not so good in the training process. Is it something that I have misunderstood or missed? Looking forward to your kind reply. Thanks!

@PerdonLiu Hi, for an image of (224*224), you use (patchsize=33, stride=8) to generate patches, you should get (24x24) logits. Why did you get (26x26) logits?