huggingface
[FEATURE] Early stopping when training
Is your feature request related to a problem? Please describe. When running the training script, there is no way to stop training early when performance plateaus.
This causes 2 problems:
- Wasting resources because we continue training as the model does not improve
- Overfitting since early stopping is a form of regularization
Describe the solution you'd like I would like to add an early-stopping feature to the training script.
This would involve a few parameters:
- The metric to monitor
- Whether to minimize or maximize that metric
- Number of steps or epochs to wait for an improvement
- The delta in the metric to determine if it is an improvement
Describe alternatives you've considered Many other CV training platforms offer early stopping. I would like to have early stopping in TIMM because I prefer this repo over others.
Hi @saad-palapa ,
you could easily modify the train.py script adding your own EarlyStopping method, it's straightforward. You just have to gather validation losses (eval_metrics, line 813), select the metric you want to monitor (top1 for instance), and use it as a stopping criteria.
You can use this simple yet effective implementation of EarlyStopping: https://github.com/Bjarten/early-stopping-pytorch/blob/master/pytorchtools.py
There you'll find a simple example on how to use it (in the "Train the Model using Early Stopping" section): https://github.com/Bjarten/early-stopping-pytorch/blob/master/MNIST_Early_Stopping_example.ipynb
The provided training script in TIMM is just a general script, you have to adapt it to your case :D
