seamless_communication
seamless_communication copied to clipboard
Fixed finetuning trainer and enable freezing layers
This fixes some issues with M4T finetuning CLI and enables freezing parts of a model before training
Example:
m4t_finetune \
--train_dataset gigaspeech/xs/xs_train_manifest.json \
--eval_dataset gigaspeech/xs/xs_validation_manifest.json \
--batch_size 10 \
--eval_steps 1000 \
--learning_rate 0.00008 \
--patience 10 \
--save_model_to checkpoints/expt5_m4tM.pt \
--model_name seamlessM4T_medium \
--freeze_layers \
model.speech_encoder_frontend \
model.speech_encoder \
model.text_encoder_frontend \
model.text_decoder.layers.0 \
model.text_decoder.layers.1 \
model.text_decoder.layers.2