dfdx icon indicating copy to clipboard operation
dfdx copied to clipboard

Padding type for Conv2d

Open Cifram opened this issue 1 year ago • 2 comments

PyTorch supports an optional parameter for 2D convolutions called padding_mode, which defaults to zeros, but can also be reflect, replicate or circular. This changes what values it uses for the padding. See the docs here: https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html

Sadly, it does not detail exactly how each of these modes works. I use replicate in my work, so I know what that does is duplicate the nearest original value in the tensor, and this is useful to avoid having the padded border appear to be an edge due to the high contrast between the zeroes and the actual border colors. Presumably, reflect mirrors the values from the edge rather than just repeating the last value. I do not know what circular does.

Cifram avatar Apr 05 '23 18:04 Cifram

Circular makes the padding wrap, so the top padding is taken from the bottom of the image and so on.

opfromthestart avatar Apr 05 '23 18:04 opfromthestart

Oh, right, makes sense that would be the other mode. I'd suggest "wrap" is a better name for that mode.

Cifram avatar Apr 05 '23 18:04 Cifram