Random Pooling causing nonzero loss for same inputs

[ShairozS]

Just a note, I was getting issues with two same inputs having nonzero loss and I tracked it to within the _random_pooling() method - the random sampling of the features can sometimes be different between images and gt. Setting torch.manual_seed(0) inside both the random pooling and sampling functions resolved this issue. I suggest adding a parameter for global seed

[HaolyShiit]

Hello! I find images and gt have the same random indices, so the random sampling of the features should be same when the inputs are same.

def _random_pooling(feats, output_1d_size=100):
    single_input = type(feats) is torch.Tensor

    if single_input:
        feats = [feats]

    N, C, H, W = feats[0].size()
    feats_sample, indices = Contextual_Loss._random_sampling(feats[0], output_1d_size**2, None)
    res = [feats_sample]

    for i in range(1, len(feats)):
        feats_sample, _ = Contextual_Loss._random_sampling(feats[i], -1, indices)
        res.append(feats_sample)

    res = [feats_sample.view(N, C, output_1d_size, output_1d_size) for feats_sample in res]

    if single_input:
        return res[0]
    return res

[HaolyShiit]

I also have a question. In the function forward(self, x) of VGG_Model.py: def forward(self, x): for index, layer in enumerate(self.vgg_model): x = layer(x) if index in self.listen: self.features[vgg_layer_inv[index]] = x return self.features

After testing I find that the outout of conv_3_2 (or other layer) is equal to the output of its next layer (after ReLU). Maybe the code should be changed to like this : def forward(self, x): for index, layer in enumerate(self.vgg_model): x = layer(x) if index in self.listen: self.features[vgg_layer_inv[index]] = x.clone() return self.features

[ShairozS]

Hmm I wonder if the problem could be in the random pooling? I'll also try to dig into it.

As for the second question, I assume this is happening when both layers are in self.listen? Yes, this happens with pointers to Tensors (but not with standard python ints). Here's a minimal example that might help

Since x is a fixed size Tensor and never bigger than any of VGG's intermediate, cloning it shouldn't cause too much harm and would probably fix this.