LwF CIFAR-100 (10/10) No exemplars accuracy

[Yuki-Hong]

Hello, I noticed that the accuracy of the LwF method in CIFAR-100 (10/10) drops to 16% after 10 tasks when the No exemplars approach is used, which is different fromthe 30.2% accuracy reported in your paper. I waswondering if you could help me understand this discrepancy? Thank you.

[mmasana]

Hi! Could you share the arguments that you used for your setting? That would make it easier to figure out the discrepancy. Maybe you didn't run the experiment with the gridsearch parameters and directly used lambda=1 for all tasks?

Here is the output for one of the seeds:

TAg Acc
	 78.8%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0% 	Avg.: 78.8% 
	 72.2%  41.9%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0% 	Avg.: 57.0% 
	 57.5%  24.3%  70.6%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0% 	Avg.: 50.8% 
	 55.8%  23.3%  70.5%  12.8%   0.0%   0.0%   0.0%   0.0%   0.0%   0.0% 	Avg.: 40.6% 
	 45.6%  12.6%  57.3%   5.7%  74.2%   0.0%   0.0%   0.0%   0.0%   0.0% 	Avg.: 39.1% 
	 41.4%  10.9%  52.9%   2.6%  68.2%  44.1%   0.0%   0.0%   0.0%   0.0% 	Avg.: 36.7% 
	 36.2%   8.8%  51.1%   1.2%  64.8%  38.0%  49.3%   0.0%   0.0%   0.0% 	Avg.: 35.6% 
	 33.0%   7.7%  49.6%   1.1%  64.3%  32.4%  45.9%  36.1%   0.0%   0.0% 	Avg.: 33.8% 
	 25.8%   4.7%  45.7%   0.9%  58.4%  26.6%  36.8%  26.0%  58.2%   0.0% 	Avg.: 31.5% 
	 21.8%   5.2%  44.1%   0.8%  57.3%  25.5%  32.9%  23.7%  58.8%  43.0% 	Avg.: 31.3% 

and the relevant/simplified arguments used:

"T" : 2,
"approach" : "lwf",
"batch_size" : 128,
"datasets" : ["cifar100_icarl"],
"gridsearch_tasks" : 10,
"nepochs" : 200,
"network" : "resnet32",
"num_exemplars" : 0,
"num_exemplars_per_class" : 0,
"num_tasks" : 10,
"pretrained" : false,
"seed" : 0,
"warmup_nepochs" : 0,
"weight_decay" : 0.0