nfnets_pytorch
nfnets_pytorch copied to clipboard
benjs
About WSconv2d
I see the authoe's code, I find his WSconv2d pad_mod is 'same'. Pytorch's conv2d dono't have pad_mode, and I think your padding should greater 0, but I find your padding always be 0. I want to know why?
I see you train.py your learning rate is constant, why? Thank you!
You are right, on first sight, the Stem layers need an additional padding='1' (all convolutions with kernel size 3).
I will take a closer look soon.
It is constant, because I did not implement something fancier yet. What do you think it should be?
Thank you!
You are right, on first sight, the Stem layers need an additional
padding='1'(all convolutions with kernel size 3). I will take a closer look soon.It is constant, because I did not implement something fancier yet. What do you think it should be?
Thank you!
when kernel size is 3, padding should be different, because stride is 1 or 2. I see the author's code learning rate is changed by warm up cosine decay. I also want to know if I train with multiple gpus, how to change batch size.
This repo does not support multi gpu training, you can only reuse the model code for that.