about initialize_weight

[LeeCASC]

I think the function of initialize_weight can be: def initialize_weights(*models): for model in models: for m in model.modules(): if isinstance(m, nn.Conv2d): nn.init.kaiming_normal_(m.weight.data, nonlinearity='relu') elif isinstance(m, nn.BatchNorm2d): m.weight.data.fill_(1.) m.bias.data.fill_(1e-4) elif isinstance(m, nn.Linear): m.weight.data.normal_(0.0, 0.0001) m.bias.data.zero_()

While using ReLU, Kaiming initializing is better.