Imene Kerboua

> waiting until #582 This is not really an issue, @loicmagne already answered, you need to adjust your dataset to meet the expected format by the PC task. Please check...

Hello, in the file there is more than one task using different versions of this dataset with different languages. A better formulation would be to make one `ScalaClassification` task that...

@dokato you can open a PR if you feel so, and also ask to join the hugging face organization to be able to create a repository on mteb and upload...

Hello @Hannibal046 , Actually, I don't see the difference between the 2 tokenizers and 2 encoders you added in your implementation. The tokenizers for queries and context are the same...

Okay my bad then. Maybe you should reopen a PR and put the results in the old folder so we can see the differences in results ?

Actually, `SentenceTransformers` proposes two models: one for queries https://huggingface.co/sentence-transformers/facebook-dpr-question_encoder-single-nq-base and one for context : https://huggingface.co/sentence-transformers/facebook-dpr-ctx_encoder-multiset-base

> Can we use these two models simultaneously for MTEB? If so, it would be very convenient to benchmark dual encoder! > > BTW, do I have to wait for...

If you're talking about Multilabel classification tasks, I think this PR https://github.com/embeddings-benchmark/mteb/pull/440 may be interesting 🙂

I guess you can create a new task using the same dataset and only change the label column? But since it's the same dataset and text, I don't know if...

I think you can frame it as a multilabel task, since the dataset offers 2 columns that can be used as labels. It's just that in a multilabel setting you'll...