mean-teacher
mean-teacher copied to clipboard
CuriousAI
what is the difference between input and ema_input
The algorithm works by having two models that take a similar input and produce a similar output: the student model, which is a normal convolutional neural network, and a teacher model, which is the same as student except its weights are exponential moving average (EMA) of the student network. Before feeding the input to the networks, the algorithm adds noise to the input. This is done for the networks separately, sampling noise twice from the same distribution.
In the Pytorch code, the sampling of noise twice is handled by TransformTwice class (for example here). The resulting transformation takes one input image and returns two noisy versions of it. The transformation is wrapped in to the train_loader, and when the train_loader is iterated (here), it returns those two noisy input images: input and ema_input. The ema_ prefix there just means that it is the one to be fed to the ema_model, i.e. the teacher network.
The algorithm works by having two models that take a similar input and produce a similar output: the student model, which is a normal convolutional neural network, and a teacher model, which is the same as student except its weights are exponential moving average (EMA) of the student network. Before feeding the input to the networks, the algorithm adds noise to the input. This is done for the networks separately, sampling noise twice from the same distribution.
In the Pytorch code, the sampling of noise twice is handled by TransformTwice class (for example here). The resulting transformation takes one input image and returns two noisy versions of it. The transformation is wrapped in to the train_loader, and when the train_loader is iterated (here), it returns those two noisy input images:
inputandema_input. The ema_ prefix there just means that it is the one to be fed to theema_model, i.e. the teacher network.
Thank you for your great answer!
