Standard recipe for fine-tuning

[desh2608]

trafficstars

Is there a recommended recipe I can refer to for fine-tuning one of the pretrained models (say, the Gigaspeech model) on my own data?

[danpovey]

There is a recipe checked in somewhere for fine-tuning a Hubert model. If it's for fine-tuning a model created by us, it would be easier to just modify the train.py:

• replace the dataset, e.g. in asr_datamodule.py
• change the learning rate schedule, to start from a lower number and decay slower, e.g. multiply initial-lr by 0.2 and increase lr-num-batches and lr-num-epochs, e.g. a factor of 2 to 4.
• start from an existing module. Could do this by just putting the pretrained model as 1.pt (assuming we didn't strip it down somehow when we uploaded it), and running with --start-epoch 2.