Ziqing Xing

icon indicating a website link https://www.zju.edu.cn/

icon company Zhejiang University icon location Zhejiang University icon location Zhejiang University/major in Information and Communication Engineering

Results 2 issues of Ziqing Xing

dropout need to set mask's device same as input Tensor!

5 comment

The FLOPS of scalar dot products in Attention has not been considered？

2 comment

When testing with the TransformerEncoderLayer, the computed FLOPS show a strictly linear relationship with the sequence length \( L \), rather than the theoretical \( L^2 \) relationship expected from...