unify-parameter-efficient-tuning icon indicating copy to clipboard operation
unify-parameter-efficient-tuning copied to clipboard

Published 20 hours ago • verified publisher icon jxhe

Does this unified view take attention mask into consideration?

Open zwbx opened this issue 2 years ago • 1 comments

I am not familiar with the theoretic derivation, but I am interested in the range of suitability of the formula。Thank you。

zwbx avatar Nov 22 '22 02:11 zwbx

Yes the derivation holds for masked attention

jxhe avatar Nov 22 '22 08:11 jxhe