mteb icon indicating copy to clipboard operation
mteb copied to clipboard

Published 20 hours ago • verified publisher icon embeddings-benchmark

Convert SIB200Classification to SIB200Clustering

Open KennethEnevoldsen opened this issue 9 months ago • 1 comments

We currently have quite a reasonable coverage on Classification tasks in MTEB, however only have a subpar performance on clustering. I would probably suggest that we convert SIB200Classification to a clustering task. @x-tabdeveloping what are your thoughts here? @jankounchained might be something that you could do? (we should probably combine train/val/test into one dataset).

KennethEnevoldsen avatar May 15 '24 15:05 KennethEnevoldsen

Sounds very reasonable to me. Topics are better suited for clustering than classification in my opinion anyway.

x-tabdeveloping avatar May 15 '24 20:05 x-tabdeveloping

Just opened a PR ☝️ if anyone wanted to take a look.

isaac-chung avatar May 20 '24 08:05 isaac-chung