Unclear how invariance to input profiles is achieved

[tsjain]

From the paper, o1 = ReLU Batch [W1 W2] R1 o2 = ReLU Batch [W2 W1] R2

If I reverse(_r) the order of R1 and R2, I get, o2_r = ReLU Batch [W1 W2] R2 o1_r = ReLU Batch [W2 W1] R1

The Hamard product q = o1 . o2 doesn't appear to be the same as q_r = o2_r . o1_r

Could you please explain how the network is invariant to the order in which the protein sequences within each pair are input?