Why does PeriodicPositionalEncoding have a dropout?

[JeremyCJM]

Hi Evelyn,

Why does PeriodicPositionalEncoding have a dropout?

Thanks, Jeremy

[EvelynFan]

Hi Evelyn,

Why does PeriodicPositionalEncoding have a dropout?

Thanks, Jeremy

Hi, this follows the default setting in the original position encoding of transformer. The class PositionalEncoding(nn.Module) has a dropout (https://github.com/pytorch/tutorials/blob/master/beginner_source/transformer_tutorial.py).