Small question: where does the core of the method happen.

[fraboeni]

Thank you for providing the repo. It is a really interesting project. I am trying to understand where the core of the method happens. It seems that conceptually, the core is to update the BatchNorm statistics of the teacher when distilling with Gaussian noise: here.

Is that already everything we need to consider, or is there something else, e.g. somewhere where we need to manually update the running statistics? I have not found anything like that in the repo.

Thanks and kind regards Franziska