Magic-NLPer icon indicating copy to clipboard operation
Magic-NLPer copied to clipboard

关于transformer模型的MultiHeadAttention函数

Open CK-IMUT-501 opened this issue 3 years ago • 2 comments

你好,在transformer的MultiHeadAttention函数中定义了self.wq、self.wk、self.wv但是在forward里仅用了self.wq来对q、k、v进行线性变换。

CK-IMUT-501 avatar Apr 06 '21 08:04 CK-IMUT-501

@CK-IMUT-501 笔误,写错了,肯定是各自用各自的wq,wk,wv,谢谢提醒

qingyujean avatar Apr 06 '21 08:04 qingyujean

@CK-IMUT-501 笔误,写错了,肯定是各自用各自的wq,wk,wv,谢谢提醒

谢谢大佬的博客,非常受用。

CK-IMUT-501 avatar Apr 06 '21 09:04 CK-IMUT-501