Imene Kerboua

Results 53 comments of Imene Kerboua

@isaac-chung this PR is ready for review. I ran both models on one language pair, but a compelete evaluation is coming (takes some time for 49 pairs). I didn't add...

@isaac-chung @Andrian0s I fixed the code to build language pair in this format: `lang1-lang2` (see screenshot below), and renamed `_eval_monolingual()` in `AbsTaskRetrieval`.I think it's ready for merging 🙂 ![image](https://github.com/embeddings-benchmark/mteb/assets/33312980/9875b0ff-2547-48c4-9d4b-796ddcd03759)

Okay I may have broken something, tests are all good locally.. I'm checking

This PR #560 will allow Retrieval tasks to handle CrossLingual tasks also

That's really intresting! Some suggestions here: - We could replace `implementation` with `framework`. - For languages, can `multilingual` be an option? We sometimes don't know how many languages a model...

A bit late, but my thoughts are: go for solution 2 and use `polars`. I think it's okay to have a language feature, it would maybe require an update of...

@shreeya-dhakal tests are not passing, can you please check locally and update?

I may be able do this, is anyone working on it? @Muennighoff @KennethEnevoldsen ? Should I wait until this PR https://github.com/embeddings-benchmark/mteb/pull/233 is merged ? I think the PR addresses BEIR...

> Parts of MIRACL are already in MTEB I think - Ideally we unify them so that it's just one MIRACL task from which the different datasets can be selected...

@crystina-z @thakur-nandan if you haven't started yet I can take it from here 😊