Motivation for x_context

[neverfox]

I am curious about the motivation for this step, as I couldn't anything about it in See et al., though I might have just missed it.

[atulkum]

Yes in the paper it is not mentioned anywhere but the code has it.

https://github.com/abisee/pointer-generator/blob/master/attention_decoder.py#L150

[atulkum]

I get the paper where similar kind of attention mechanism is used.

Order Matters: Sequence to sequence for sets

[lorenzoale]

Thanks a lot for this work! I have a question about when do you update this context. During training, as far as I understand, seems to be ok. However, during decoding, in the abisee code they seem to update the context before using it in lstm: https://github.com/abisee/pointer-generator/blob/a7317f573d01b944c31a76bde7218bcfc890ef6a/attention_decoder.py#L140 And then also update it after the call to lstm. https://github.com/abisee/pointer-generator/blob/a7317f573d01b944c31a76bde7218bcfc890ef6a/attention_decoder.py#L158

[atulkum]

Thanks for pointing this out. You are right. I have updated my code, I still need to re-run the experiments though. I will update the result after that. Here is the commit: https://github.com/atulkum/pointer_summarizer/commit/454a2f6f10865531f724cbe5064a927d66dfa1b7 By the way I am thinking about experimenting with transformer network for this task and verify speed and accuracy tradeoff. Let me know if you want to collaborate.