avalanche icon indicating copy to clipboard operation
avalanche copied to clipboard

CORe50 Dataset and Benchmark Improvements

Open vlomonaco opened this issue 3 years ago • 1 comments

We should:

  • [x] Expand category-level classification support.
  • [x] Add CORe50 mini (32x32)
  • [ ] Test set split in experiences option
  • [ ] Document the dataset and benchmarks better.
  • [ ] Add default transformations
  • [ ] Add segmentation and detection support.

vlomonaco avatar Apr 20 '21 08:04 vlomonaco

Additionally I would like to see a flag for using a subsampled test set. I've found Core50 test set to be needlessly large. In fact the Core50 authors end up sub-sampling it in later works.

"we sub-sampled the test set by selecting 1 frame every second (from the original 20 fps)"
Lomonaco, V., Maltoni, D., & Pellegrini, L. (2020). Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches. ArXiv:1907.03799 [Cs, Stat]. http://arxiv.org/abs/1907.03799

Additionally some sort of meta-loader that wraps any benchmark allowing sampling to be applied to any dataset might be useful.

tachyonicClock avatar Nov 10 '21 23:11 tachyonicClock