I have not seen any differences between m1 and m2, Why?

[tjpulkl]

part of your code: m1 = nn.Sequential( nn.Conv2d(3, 3, 1, 1, bias=False), nn.BatchNorm2d(3), nn.ReLU(inplace=True), nn.Conv2d(3, 3, 1, 1, bias=False), nn.BatchNorm2d(3), ).cuda() torch.manual_seed(123) init_weight(m1) m2 = nn.Sequential( nn.Conv2d(3, 3, 1, 1, bias=False), NN.BatchNorm2d(3), nn.ReLU(inplace=True), nn.Conv2d(3, 3, 1, 1, bias=False), NN.BatchNorm2d(3), ).cuda() result: m1(nn.BatchNorm2d) running_mean tensor([-0.0488, 0.3387, 1.2459], device='cuda:0') tensor([ 0.2472, 0.7088, -0.1562], device='cuda:0') m2(NN.BatchNorm2d) running_mean tensor([-0.0488, 0.3387, 1.2459], device='cuda:0') tensor([ 0.2472, 0.7088, -0.1562], device='cuda:0') m1(nn.BatchNorm2d) running_var tensor([0.2357, 0.0488, 0.2370], device='cuda:0') tensor([0.1876, 0.5313, 1.2456], device='cuda:0') m2(NN.BatchNorm2d) running_var tensor([0.2357, 0.0488, 0.2370], device='cuda:0') tensor([0.1876, 0.5313, 1.2456], device='cuda:0') m1(nn.BatchNorm2d) weight Parameter containing: tensor([1.0040, 0.9928, 1.0031], device='cuda:0', requires_grad=True) Parameter containing: tensor([0.9896, 0.9912, 0.9916], device='cuda:0', requires_grad=True) m2(NN.BatchNorm2d) weight Parameter containing: tensor([1.0040, 0.9928, 1.0031], device='cuda:0', requires_grad=True) Parameter containing: tensor([0.9896, 0.9912, 0.9916], device='cuda:0', requires_grad=True) m1(nn.BatchNorm2d) bias Parameter containing: tensor([ 0.0023, -0.0054, 0.0019], device='cuda:0', requires_grad=True) Parameter containing: tensor([0.0645, 0.0645, 0.0645], device='cuda:0', requires_grad=True) m2(NN.BatchNorm2d) bias Parameter containing: tensor([ 0.0023, -0.0054, 0.0019], device='cuda:0', requires_grad=True) Parameter containing: tensor([0.0645, 0.0645, 0.0645], device='cuda:0', requires_grad=True) Thank you!

[boykac]

i have the same question

[YiLiM1]

i have the same question