Forgetting metric for first task after learning two tasks is 0

[Kishaan]

Hi,

I checked the logs_1.json file created after finishing training on cifar-100 10 tasks and I see forgetting metric to be 0.0. There would be some forgetting for the first task data after learning the second task. I checked the code for calculating this metric and I see the main loop which calculates forgetting metric does not run after learning second task:

for j in range(k - 2): # after learning two tasks, k would be 2 here

I would like to know if this is expected or if this is a bug. Forgetting metric is important for my study. I would like to get it right.

[arthurdouillard]

Hum, which continuum version do you have installed?

[Kishaan]

1.2.4

[Kishaan]

To clarify my question, I do get a number for forgetting metric in the log after training 10 tasks, my question is about the forgetting metric after learning the second task.

[arthurdouillard]

It's probably worth opening an issue about it in continuum, Timothe lesort can get a look at this.