attention-is-all-you-need-pytorch icon indicating copy to clipboard operation
attention-is-all-you-need-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon jadore801120

n_position in positional encoding

Open Tejaswini2612 opened this issue 4 years ago • 1 comments

How did you choose n_position as 200. What is this number based on?

Hoping to hear back. Thank you!

Tejaswini2612 avatar Mar 05 '20 21:03 Tejaswini2612

I think n_position is used to generate sin(x) and cos(x) value list, then slice [:SL], so only need n_position >= SL (seq_len)

alphaRGB avatar Sep 25 '20 05:09 alphaRGB