How is the feature map visualization got?

[XiyueSun]

how to do the feature map visualization like figure 2 and 7?

[namepllet]

We apply absolute sum to make channel dimension 1, and show feature map from matpoltlib package.

[jingyushi]

@namepllet Hi, Park and Ou. Thanks for your impressive work. Could please also let me know how you obtained the Correlation Map as in Figure 4? I have now the 1x4x1024x1024 shape C from the matmul of Csoft and Csig. Thanks in advance!

[namepllet]

The correlation map in figure 4 shows correlation between single query(red point) and other keys.

So you can choose any query and reshape key dimension to 32x32 to visualize correlation map.

[karta2155802]

Hi, I'm trying to visualize the primary feature map like figure 2 by following your method above with the demo model weight. But I couldn't get the same visualization as yours. Here's my code, input image and visualization results. Which part I did wrong?

        pfeats = primary_feats.abs().sum(dim=1).squeeze()

Thanks

[namepllet]

Since we refactored our code after paper submission, the feature map may look different.

And for the HO3D images, use pretrained model weight for HO3D. (The demo model is not trained on HO3D)