DeepMultiCap icon indicating copy to clipboard operation
DeepMultiCap copied to clipboard

Question about your attention module

Open sta105 opened this issue 2 years ago • 1 comments

Hi,

In DMCNet, You firstly use Hourglass to extract multi-level features, then you fuse these features from differnt views by attention module.

However, the code shows below indicates you discard all the low level features and only uses the last level. Which is different from the original implentation of PIFu. Is it right?

def attention(self, feat, feature_fusion):
    att_feat = torch.zeros_like(feat[-1:])
    num_views = self.num_views
    
    for view in range(num_views):
        att_feat[-1, :, view] = feat[-1, :, view]  ## Only use the last level ?
    att_feat = att_feat.permute(0, 1, 4, 2, 3).contiguous().reshape(-1, num_views, feat.shape[3])
    att_feat, = feature_fusion(att_feat)
    _, B, V, D, N = feat.shape
    att_feat = att_feat.reshape(-1, B, N, V, D).permute(0, 1, 3, 4, 2)
    return att_feat

sta105 avatar Mar 14 '22 09:03 sta105

Hi,

Thanks for your advice. We take the last level feature since we only have one attention module. It may be more efficient to build several attention modules on features of each level. You can have a try.

Hi,

In DMCNet, You firstly use Hourglass to extract multi-level features, then you fuse these features from differnt views by attention module.

However, the code shows below indicates you discard all the low level features and only uses the last level. Which is different from the original implentation of PIFu. Is it right?

def attention(self, feat, feature_fusion):
    att_feat = torch.zeros_like(feat[-1:])
    num_views = self.num_views
    
    for view in range(num_views):
        att_feat[-1, :, view] = feat[-1, :, view]  ## Only use the last level ?
    att_feat = att_feat.permute(0, 1, 4, 2, 3).contiguous().reshape(-1, num_views, feat.shape[3])
    att_feat, = feature_fusion(att_feat)
    _, B, V, D, N = feat.shape
    att_feat = att_feat.reshape(-1, B, N, V, D).permute(0, 1, 3, 4, 2)
    return att_feat

DSaurus avatar Mar 16 '22 08:03 DSaurus