arctic-captions question about grads of alphas in hard Attention

question about grads of alphas in hard Attention

Open denglixi opened this issue 8 years ago • 4 comments

Hello, I feel realy confused about the grads of alphas in hard attention. The source code is in line 1199:

known_grads={alphas:opt_outs['masked_cost'][:,:,None]/10.* (alphas_sample/alphas) + alpha_entropy_c*(tensor.log(alphas) + 1)})

Can anyone explain this to me, please?

Mar 04 '16 01:03 denglixi

@denglixi I am also confused by this. Did you find the answer？

Dec 08 '16 06:12 ysjakking

@denglixi @ysjakking Did you figure out the answer? I am also confused. Can anyone help me?

Sep 26 '17 07:09 SijieSong

Me too...

Jan 19 '18 13:01 shaoxuan92

@denglixi @ysjakking @shaoxuan92 @SijieSong Any solution did you get? I also come across this problem.

Jun 16 '19 10:06 AlvinAi96