inquiry on attention part

[Derekkk]

Hi there,

Thanks for sharing the code. For attention part in model.py, your code is:

attentionSoft_b = tf.nn.softmax(tf.transpose(attentionWeights))
attentionSoft_b = tf.transpose(attentionSoft_b) 

while I feel like it should be: attentionSoft_b = tf.nn.softmax(attentionWeights, axis=1)

or you should indicate the "perm" in transpose function.

Please correct me if I'm wrong, thanks!

[randomtutu]

u r right . tf.transpose() would trans all tha axis of the tensor. such as [4,3,2] ——>[2,3,4]