Shuai Li

When both "batch_norm_use_averages" and "batch_norm_update_averages" are enabled, the mean and inv_std are obtained from the past data. For example, when training the second batch, mean and inv_std are from the...

First, I think it is the right way to perform batch normalization when enabling "batch_norm_use_averages ". Second, when I use BN for RNN, it makes a big difference because the...

Okay. Thanks. I will let you know when I have the final results.