Wrong implementation about EMA ?

[YilinLiu97]

Hi @perone, I found that the teacher model's weights seem to be not updated as it performed as bad as it was first initialized.

def update_ema_variables(model, ema_model, alpha, global_step):

  alpha = min(1 - 1 / (global_step + 1), alpha)
  for ema_param, param in zip(ema_model.parameters(), model.parameters()):
  ema_param.data.mul_(alpha).add_(1 - alpha, param.data)

About this

line:ema_param.data.mul_(alpha).add_(1 - alpha, param.data),

shouldn't this be ema_param.data.mul_(alpha).add_((1 - alpha)*param.data) ?? Thanks!