Bug report

[pren1]

Please change this part:

train_mean_op = tf.assign(pop_mean, pop_mean * decay + batch_mean * (1 - decay))
train_var_op = tf.assign(pop_var, pop_var * decay + batch_var * (1 - decay))
def batch_statistics():
    with tf.control_dependencies([train_mean_op, train_var_op]):
        return tf.nn.batch_normalization(x, batch_mean, batch_var, offset, scale, epsilon)

to:

 def batch_statistics():
    train_mean_op = tf.assign(pop_mean, pop_mean * decay + batch_mean * (1 - decay))
    train_var_op = tf.assign(pop_var, pop_var * decay + batch_var * (1 - decay))
    with tf.control_dependencies([train_mean_op, train_var_op]):
        return tf.nn.batch_normalization(x, batch_mean, batch_var, offset, scale, epsilon)

According to the problem at stack overflow at here. Otherwise, the train_mean_op and train_var_op will always be executed despite the value of training.

By the way, this implementation does not work well in my case, sad thing lol