Imene Kerboua

Okay, so I think the easiest solution is to allow re-runing the jobs at least one, it may fix this issue.

> I tried running the intfloat/multilingual-e5-small but the run got terminated. Any reason why that might be the case? Maybe a problem with your hardware? Can you copy paste the...

@Akash190104 any news on this PR?

I personally find it weird to have classes with only 1 sample, maybe we shouldn't handle them? We can filter the dataset and remove rows with only 1 sample, wdyt?...

Hello @ManuelFay, thanks for the comments I'll try to answer some and review the PR: > - AlloprofRetrieval task has an updated dataset so it is required to modify code...

> @imenelydiaker will you guys make a PR with the changes when you have the time? Yes I'll open a PR soon. > It seems like a good idea to...

It may be due to the training of Solon and Sentence-croissant-alpha. The other possibility can be that other models either have been trained on the test set, or the train...

I was thinking about improving the French benchmark by adding the [FQuad](https://huggingface.co/datasets/fquad) [[paper](https://arxiv.org/abs/2002.06071)] QA dataset, it can be used for retrieval tasks. @ManuelFay would you be interested in helping? (I'm...

@ManuelFay: Regarding the Pair classification task with OpusParcus dataset, the results are updated [here](https://huggingface.co/datasets/mteb/results/discussions/41). The problem was as mentionned in https://github.com/embeddings-benchmark/mteb/pull/342, the use of the wrong `test` split to run...

Issues to fix: - [x] Removing BitextMining from fr evaluation script. - [x] Updating OpusParcus results. - [x] Fix split name of AlloprofRetrieval and SyntecRetrieval. - [ ] What to...