chainer_examples icon indicating copy to clipboard operation
chainer_examples copied to clipboard

Published 20 hours ago • verified publisher icon odashi

Attention Mechanism is Very Slow

Open prajdabre opened this issue 9 years ago • 2 comments

Hi, Your attention mechanism is quite slow. Since you compute the linear projections (aw and bw) each time although they do not change, the time is almost quadratic.

I have implemented a faster version of attention which does a lot of precomputation and would like to push it as soon as I am done testing.

Regards.

prajdabre avatar Feb 08 '16 09:02 prajdabre

Thank you, I had overlooked this issue and I think it is no problem to fix about that calculation.

Regards,

odashi avatar Feb 14 '16 13:02 odashi

Hi, I have implemented the fast version of attention and have tested it. I will push it tomorrow or so.

prajdabre avatar Feb 14 '16 14:02 prajdabre