pointer-generator icon indicating copy to clipboard operation
pointer-generator copied to clipboard

Published 20 hours ago • verified publisher icon abisee

NAN source?

Open caoxu915683474 opened this issue 5 years ago • 1 comments

https://github.com/abisee/pointer-generator/blob/b29e986f24fdd01a6b6d6008187c5c887f0be282/attention_decoder.py#L101

caoxu915683474 avatar Dec 17 '19 07:12 caoxu915683474

return attn_dist / (tf.reshape(masked_sums, [-1, 1]) + tf.ones_like(tf.reshape(masked_sums, [-1, 1])) * sys.float_info.epsilon)

tf.reshape(masked_sums, [-1, 1]) seems will not be zeros during training, but it is also harmful due to "division by 0", NAN issue may be caused by this reason.

caoxu915683474 avatar Dec 17 '19 07:12 caoxu915683474