MLMAN
MLMAN copied to clipboard
请问能解释一下全连接层和lstm这样初始化的原因吗?
全连接层初始化代码
bias = np.sqrt(6.0 / (input_linear.weight.size(0) + input_linear.weight.size(1))) nn.init.uniform_(input_linear.weight, -bias, bias) if input_linear.bias is not None: input_linear.bias.data.zero_()
lstm层初始化代码
for ind in range(0, input_lstm.num_layers): weight = eval('input_lstm.weight_ih_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias) weight = eval('input_lstm.weight_hh_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias)