contrib icon indicating copy to clipboard operation
contrib copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

[question] SWA BatchNorm momentum reset

Open vadimkantorov opened this issue 6 years ago • 0 comments

Could you please comment on the particular way of decaying the BatchNorm momentum parameter for every mini-batch during BatchNorm parameters update
https://github.com/pytorch/contrib/blob/master/torchcontrib/optim/swa.py#L305 ?

(As far as I understand, BatchNorm momentum is usually constant)

Thanks!

vadimkantorov avatar Nov 14 '19 13:11 vadimkantorov