Invalid reference to variable

[steffenvan]

Hi,

Thanks for putting the code from the paper into a file. I'm wondering which value is actually supposed to be reshaped in these lines.

def augmented_conv2d(X, Fout, k, dk, dv, Nh, relative):
...
    attn_out = tf.matmul(weights, flat_v)
    attn_out = tf.reshape(v, [B, Nh, H, W, dv // Nh])
    attn_out = combine_heads_2d(v)
...

v is a variable unreachable/undefined in the current scope, so I'm wondering if this is intentional?

Edit: Perhaps the correct variable should be flat_v?

[pquochuy]

I am also wondering. But it guess, v should be attn_out, and B is the batch_size (the first dimension of X).