pyannote-audio
pyannote-audio copied to clipboard
pyannote
Wespeaker embeddings question
If I wanted to use one of the larger wespeaker models - say 293 - would I just download the .pt file and point to it in the config.yaml?
It is a tiny bit more complex than that.
See this script that does most of the job.
If you work on this, would be nice to share them on Huggingface, taking this repo as example.
I have never uploaded a model to huggingface before. Is there some way I can give it a similar name - like pyannote/wespeaker-voxceleb-resnet293-LM? If I understand correctly, the way the code works is that it first scans for the keyword "pyannote" in the model name, so another option I assume would be to call it "picheny/pyannote-wespeaker-voxceleb-resnet293-LM". Another concern is that the use of these embeddings is not giving me any improvement on my task (relative to the 34M version). That could just be life, or I might have messed something up.......
It should be fine with picheny/wespeaker-voxceleb-resnet293-LM since it will end up using this branch of the code:
https://github.com/pyannote/pyannote-audio/blob/66dd72bb2b807aaf6d011c89678d85b51fb3b859/pyannote/audio/pipelines/speaker_verification.py#L764-L768
Also, in my speaker diarization experiments, larger models did not bring any significant (or consistent) improvement either. That is why I sticked with the ResNet34 version for pyannote/speaker-diarization-3.1.
Larger models might help for speaker verification, though.
@hbredin Thanks for the awesome work!
I want to ask if I need to change the clustering threshold when using wespeaker-voxceleb-resnet293-LM? If so, could you please share the experimental threshold you used when testing the resnet293-LM?
Yes, you would need to optimize thresholds for each version of the embedding network. However, I did not keep track of the optimized thresholds, sorry.
@hbredin Thanks for the quick response. Can you please point me out some documentation or some guidelines to set the threshold value? And using which dataset, I should perform benchmarking.
Grid search should be fine. For benchmarking, I guess you’d have to use data similar to the expected test/production data.
@hbredin Currently, I dont have any annotated data. Can you inform that the existing setup is on which dataset? The current configurations are working fine on my data.