mteb
mteb copied to clipboard
Problems with mteb_meta for german evaluation
Hi everyone, I am having troubles with generating the mteb_meta for German, with just running the script.
I am currently trying to format results but it does not seem to work straight away with "mteb_meta.py" - any idea? I just get a blank metadata file:
tags:
mteb model-index: name: gbert-large
results:
it gives this for every dataset:
WARNING:mteb.evaluation.MTEB:Passing task names as strings is deprecated and will be removed in the next release. Please use tasks = mteb.get_tasks(tasks=[...]) method to get tasks instead.
INFO:main:Skipping AmazonCounterfactualClassification as split test not present.
WARNING:mteb.evaluation.MTEB:Passing task names as strings is deprecated and will be removed in the next release. Please use tasks = mteb.get_tasks(tasks=[...]) method to get tasks instead.
INFO:main:Skipping AmazonReviewsClassification as split test not present.
Do I need to modify something on the code?