Problem with Wav2Vec finetune task

[ayw001]

Last year with fairseq, I successfully finetuned both base and large model with my own data. The result was very good. However, I run into the problem when I try to repeat the same finetune task. At the beginning, the training seems normal as eval_loss decline as expected but eval_wer is still 1.0. But eventual, valid_loss go to 'nan' and the output model does not produce anything. One thing I observed in log file is ctc_loss_reduction": "sum" According to your blog, that should be "mean".

I do not configure that at all.

Is this a problem? If it is, how do I fix it.

Thanks

Yuanjun