whisper-finetune
whisper-finetune copied to clipboard
vasistalodagala
Information On Batch Size And Learning Rate
The discord link in the README does not work for me.
Do you have any information on what batch size or learning rate to use? I could only find the max learning rate that was used in the paper. Experimentally, I found that too small of a batch size seems to cause issues
What batch size and learning rate do you recommend and why?
The discord link is active. You might probably need to join the HuggingFace discord server and the ML-4-AUDIO channel within it to gain access. The discussion about learning rates could be found in the same thread. The max learning rates used in the paper can be found from the very final page.
Regarding batch size, it highly depends on the GPUs being used and their memory capacity. While higher batch sizes are recommended, one needs to experiment and fixate on the batch size which gives maximum utilization of GPU memory while not leading to out-of-memory issues. It is to be noted that the batch size in the scripts of this repository refer to the per-GPU batch size.
Thanks for great work Was the batch size mentioned here https://huggingface.co/vasista22/whisper-kannada-medium
Total or per gpu?
@Theodotus1243 , the batch size for the above mentioned model is the per GPU batch size.
Thanks How much gpus have you used, to properly reproduce your experience
@Theodotus1243 , 8 A-100 GPUs have been used for about a week to train the mentioned model
