joonson
labels for the test set
The labels for the test set will be released to public after the VoxCeleb Speaker Recognition Challenge in October 2020. How can we download the labels for the test set? Looking forward to your reply.
Thanks @JaesungHuh for sharing the test set labels.
For comparison with official results reported here, can you please confirm that these were computed on the subset of 232 files for which labels are available and not on the whole set of 312 files shared initially? This is important for the speaker diarization community to make sure we are not comparing apples and oranges.
cc @fnlandini @desh2608
@hbredin Thanks for the question. Yes, the released Voxconverse test set are subset of 232 files from the whole set of 312 files shared initially. We did another few rounds of check to make labels more accurate and removed some files which annotators couldn't be 100% sure of their annotation. Please use this version from now on.
Thanks for clarifying.
What should we call this version in publications: VoxConverse 2021 ? VoxConverse v0.0.2?
I'll re-open this issue for other people to see. I have to discuss co-authors about this, but I think either is fine. Will let you know if the term fixed.
We've recently released ver 0.3, fixing some of the errors in the test set labels. Please call "VoxConverse 0.3" when you use this dataset.
Thanks for the heads-up @JaesungHuh.
Switching reference labels from 0.2 to 0.3 did "improve" my baseline by a whooping 2.8% (relative) in terms of speaker confusion rate. That is not negligible.
Yes. We found these errors during the preparation for this year's VoxSRC workshop. I'll re-open this issue to let everyone know about this. I apologize for any inconvenience.
