seamless_communication icon indicating copy to clipboard operation
seamless_communication copied to clipboard

Train speech language ID classification head

Open am831 opened this issue 9 months ago • 0 comments

Implements a new training script for training a classification head. The new nn.module in model.py is separate from other components of M4T and takes speech encoder output and maps it into language probabilities with softmax. All layers of M4T are frozen to prevent training of those components. A new dataset.py script is implemented for downloading common voice from hugging face.

Most recent results from training:

Samples in train dataset:

- eng - 647 samples
- fra - 676 samples
- deu - 862 samples
- rus - 775 samples
- spa - 908 samples
- hin - 418 samples
- Total - 4286 samples

Samples in eval dataset:

- eng - 394 samples
- fra - 289 samples
- deu - 363 samples
- rus - 356 samples
- spa - 408 samples
- hin - 239 samples
- Total - 2049 samples

Parameters:

- batch_size 32
- learning_rate .001
- patience 10
- num_layers 4

441325260_8377882475560539_6518711659633650790_n

I am collaborating with @zrthxn on this project

am831 avatar May 09 '24 19:05 am831