avalanche
avalanche copied to clipboard
ContinualAI
Checkpointing support
This PR adds support for checkpointing.
Checkpointing is implemented as a strategy plugin that will store a checkpoint after each evaluation phase.
The entire strategy object is stored, as well as the state of random number generators.
Particular custom procedures have been implemented to manage loggers and a couple of torchvision datasets.
An example is provided which includes a description of the steps required to add checkpointing in the main script.
This PR also introduces a fix in data_loader.py when persistent_workers is used and adds the download progress bar for inaturalist.
Apart from minor git merge issues, the checkpointing functionality seems to be working with the new datasets!
- I also tested other plugins such as LwF, GDumb, EWC, CWR (only replay is tested by the CI)
- Simplified the test procedure to run on the usual fake benchmark.
Pull Request Test Coverage Report for Build 3322127543
- 85 of 218 (38.99%) changed or added relevant lines in 21 files are covered.
- 7 unchanged lines in 6 files lost coverage.
- Overall coverage decreased (-0.2%) to 72.903%
| Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
|---|---|---|---|
| avalanche/benchmarks/classic/ccifar10.py | 2 | 3 | 66.67% |
| avalanche/benchmarks/classic/cfashion_mnist.py | 1 | 2 | 50.0% |
| avalanche/training/plugins/gss_greedy.py | 0 | 1 | 0.0% |
| avalanche/benchmarks/classic/cmnist.py | 2 | 4 | 50.0% |
| avalanche/benchmarks/datasets/external_datasets/mnist.py | 13 | 15 | 86.67% |
| avalanche/logging/tensorboard_logger.py | 7 | 9 | 77.78% |
| avalanche/benchmarks/classic/ccifar100.py | 3 | 6 | 50.0% |
| avalanche/models/generator.py | 0 | 3 | 0.0% |
| tests/test_cifar100_benchmarks.py | 1 | 5 | 20.0% |
| tests/test_cifar10_benchmarks.py | 1 | 5 | 20.0% |
| <!-- | Total: | 85 | 218 |
| Files with Coverage Reduction | New Missed Lines | % |
|---|---|---|
| avalanche/logging/wandb_logger.py | 1 | 23.01% |
| tests/test_cifar100_benchmarks.py | 1 | 18.82% |
| tests/test_cifar10_benchmarks.py | 1 | 26.09% |
| tests/test_fmnist_benckmarks.py | 1 | 28.21% |
| tests/test_mnist_benckmarks.py | 1 | 18.99% |
| avalanche/benchmarks/scenarios/classification_scenario.py | 2 | 86.31% |
| <!-- | Total: | 7 |
| Totals | |
|---|---|
| Change from base Build 3297161180: | -0.2% |
| Covered Lines: | 13140 |
| Relevant Lines: | 18024 |
