Question about Relative-Position-Representation code. Thank you very much.

[guotong1988]

In https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/layers/common_attention.py

x_t_r shape is [sequence_length, batch_size, hidden_size] z shape is [sequence_length, sequence_length, hidden_size]

then tf.matmul x_t_r and z to get a tensor with shape [batch_size, sequence_length, sequence_length], How To Understand This Line? How To Understand The Insight of This Line?

To be brief, I remove the heads in shape.