firefox-translations-training icon indicating copy to clipboard operation
firefox-translations-training copied to clipboard

Support training continuation for student models

Open gregtatum opened this issue 10 months ago • 3 comments

We currently support training continuation for back translations, and teacher training.

gregtatum avatar Apr 05 '24 20:04 gregtatum

This might be required to enable pre-emption for student models. cc @bhearsum

eu9ene avatar May 08 '24 23:05 eu9ene

IIRC, this bug is about training continuation integrated with the training config.

I did just post a patch (#580) to automatically continue training for an earlier run in the same task though. The mechanism is a bit different, but I think that's the part we need to turn on spot instances again?

bhearsum avatar May 08 '24 23:05 bhearsum

If we support continuation on preemption we can close this as we don't plan to use manual training continuation for the students now.

eu9ene avatar Jun 25 '24 18:06 eu9ene