FNet-pytorch

FNet-pytorch copied to clipboard

Masking/padding tokens in sequences ?

1

How can we mask/pad tokens for sequences of varied length ? When we use fft along dimension (-2) , sequences , if we just use zero padding the result will...

d5555

Dear @rishikksh20 , Thanks for your implementation! Can I ask what do you think about the effectiveness of Furier features as well as FNet when it's used to replace multi-head...

manhph2211

Layer Normalization

3

Hi, thanks for a great implementation! I wanted to clarify one thing that mismatches with the code, proposed in the article itself. In your code, you pre-normalize inputs, so that...

Aktsvigun

are you ready to Implement embedding part?

1

thanks for your impl, and I found that you may not code emdedding part, I think the part is important too. I suggest that you could write a demo from...

BinchaoPeng

Hi, I propose you a slight factorisation of the code and it is clearer to read.

AdilZouitine