Imene Kerboua
Imene Kerboua
@isaac-chung this PR is ready for review. I ran both models on one language pair, but a compelete evaluation is coming (takes some time for 49 pairs). I didn't add...
@isaac-chung @Andrian0s I fixed the code to build language pair in this format: `lang1-lang2` (see screenshot below), and renamed `_eval_monolingual()` in `AbsTaskRetrieval`.I think it's ready for merging 🙂 
Okay I may have broken something, tests are all good locally.. I'm checking
This PR #560 will allow Retrieval tasks to handle CrossLingual tasks also
That's really intresting! Some suggestions here: - We could replace `implementation` with `framework`. - For languages, can `multilingual` be an option? We sometimes don't know how many languages a model...
A bit late, but my thoughts are: go for solution 2 and use `polars`. I think it's okay to have a language feature, it would maybe require an update of...
@shreeya-dhakal tests are not passing, can you please check locally and update?
I may be able do this, is anyone working on it? @Muennighoff @KennethEnevoldsen ? Should I wait until this PR https://github.com/embeddings-benchmark/mteb/pull/233 is merged ? I think the PR addresses BEIR...
> Parts of MIRACL are already in MTEB I think - Ideally we unify them so that it's just one MIRACL task from which the different datasets can be selected...
@crystina-z @thakur-nandan if you haven't started yet I can take it from here 😊