mi-prometheus
mi-prometheus copied to clipboard
Trainers should indicate if loss criterion is met but curriculum learning still going
The current implementation of curriculum learning forces by default that it needs to be finished even if convergence has been reached (in terms of loss < threshold):
# If the 'must_finish' key is not present in config then then it will be finished by default self.params['training']['curriculum_learning'].add_default_params({'must_finish': True})
While I think this is okay, it'd be great if the trainer would indicate that. For instance, if the loss threshold for a run of MAES on SerialRecall is set to 1e-2, MAES will most likely converge before the end of curriculem learning. In this case, a warning message would be great, in the lines of:
if not self.curric_done and converged:
self.logger.warning('The model has converged but curriculum has been set with must_finish=True.')