Training does not converge after joining compact bilinear layer

[roseif]

Source code： x = self.features(x) #[4,512,28,28] batch_size = x.size(0) x = (torch.bmm(x, torch.transpose(x, 1, 2)) / 28 ** 2).view(batch_size, -1) x = torch.nn.functional.normalize(torch.sign(x) * torch.sqrt(torch.abs(x) + 1e-10)) x = self.classifiers(x) return x my code: x = self.features(x) #[4,512,28,28] x = x.view(x.shape[0], x.shape[1], -1) #[4,512,784] x = x.permute(0, 2, 1) #[4,784,512] x = self.mcb(x,x) #[4,784,512] batch_size = x.size(0) x = x.sum(1) #对于二维来说，dim=0，对列求和；dim=1对行求和；在这里是三维所以是对列求和 x = torch.nn.functional.normalize(torch.sign(x) * torch.sqrt(torch.abs(x) + 1e-10)) x = self.classifiers(x) return x

The training does not converge after modification. Why? Is it a problem with my code?

[CHTsuperman]

Have you solved it? Can you share it？

[roseif]

Have you solved it? Can you share it？ The learning rate setting maybe too high. You can lower it and try again.

[CHTsuperman]

Have you solved it? Can you share it？ The learning rate setting maybe too high. You can lower it and try again.

Thank you! i will try it