hierarchical-attention-networks icon indicating copy to clipboard operation
hierarchical-attention-networks copied to clipboard

Published 20 hours ago • verified publisher icon ematvey

Mask for attention weight

Open 2g-XzenG opened this issue 6 years ago • 0 comments

Hi ematvey,

Thanks for sharing the code!

I notice the attention weights for sentence & word are not mask according to their actual length, which means the model will "pay attention" to the useless input. Is there a reason you didn't use a mask for the project?

Please correct me if I am wrong. Thanks! Xianlonb

2g-XzenG avatar Jun 14 '18 18:06 2g-XzenG