Snapshot-Ensembles
Snapshot-Ensembles copied to clipboard
Update per batch, not per epoch
First of all, thank you very much for sharing your implementation of the Snapshot Ensembling. However, you should not use the callbacks.LearningRateScheduler to update the learning rate.
As mentioned in the original paper under the equation 2, "we update the learning rate at each iteration" (i.e batch) "rather than at every epoch" to improve the convergence of short cycles. But, callbacks.LearningRateScheduler in Keras, with the source code here, updates the learning rate at 'on_epoch_begin'.
After this fix, you should improve further your scores.