fnet-pytorch icon indicating copy to clipboard operation
fnet-pytorch copied to clipboard

Masking/padding tokens in sequences ?

Open d5555 opened this issue 2 years ago • 0 comments

How can we mask/pad tokens for sequences of varied length ?
When we use fft along dimension (-2) , sequences , if we just use zero padding the result will be skewed. torch.fft.fft(torch.fft.fft(hidden_states.float(), dim=-1), dim=-2).real

d5555 avatar Jul 29 '22 01:07 d5555