questions about the computation graph of the nonlocal module

[Swocky]

Hello, I've read your cola-net recently, and I'm very interested in the nonlocal module. But in the study of your code, I found some differences from the paper, which may be due to memory and computing optimization. In short, the approximate calculation order in this paper is------theta: (N, CWH) * pi: (CWH,N) * g: (N, CWH) In your code--------g: (N2, CWH) * theta: (CWH, N) * pi: (N, CWH) I can understand the former from the perspective of patch match, pi and theta generate a matrix of (N * N), which is fed back to the output of g. But in the latter, you not only exchange the order, but also adjust the number of patches of g by stride. From the perspective of association law, the latter two terms are also generated (CWH, CWH). Please explain the equivalence between this calculation and the previous calculation. And why do softmax after multiplying the output of g by that of theta, thank you!

[MC-E]

Thanks for your interest. It is mainly designed to reduce the computational complexity. The former from can be viewed as using N^2 patch to update each patch. The latter (our implement) adjusts the number of patches (about 1/4*N^2) by stride to update each patch. But I do not full understand what the "exchange the order" means. The similarity calculation and updating are one-to-one correspondence by matrix multiplication.