scandinavian-embedding-benchmark
scandinavian-embedding-benchmark copied to clipboard
Find a solution allowing empty results
Some models such as the "translate and embed" models can't be used for cross-lingual tasks, ideally their scores should just be nan.
I am unsure what the best solution is. We could naturally raise an error (but then it seems like it failed). We could also return nan (but then we would have to handle it in mteb, which is also
Seems like the best solution is to raise a specific error something like ModelNotValidForThisTaskError
@x-tabdeveloping, do you have a suggestion here?