How to adapt model zoo for tasks with varying number of classes?

[appledora]

I am trying to adapt the modelzoo's minimal notebook to setup some experiments. Unlike the notebook, I am trying to use completely separate datasets as different tasks. Which means the number of classes per task is not the same.

Do you have any suggestions or pointers on how this can be done? I am currently exploring the multihead.py script, because that seems to be aligned with my goals - but I just wanted to be sure whether this might lead to any implicit error in calculations.

Thanks!