opencompass icon indicating copy to clipboard operation
opencompass copied to clipboard

FileNotFoundError: Couldn't find a module script at /data/opencompass/bleurt/bleurt.py. Module 'bleurt' doesn't exist on the Hugging Face Hub either.[Bug]

Open Modas-Li opened this issue 1 year ago • 0 comments

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

1

Reproduces the problem - code/configuration sample

1

Reproduces the problem - command or script

1

Reproduces the problem - error message

Traceback (most recent call last): File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 356, in inferencer.run() File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 76, in run self._score() File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 185, in _score result = icl_evaluator.score(**preds) File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 127, in score results = self.basic_score(predictions, references) File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 134, in basic_score metrics = {key: evaluate.load(key) for key in self.metrics} File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 134, in metrics = {key: evaluate.load(key) for key in self.metrics} File "/data/Anaconda3/envs/opencompass/lib/python3.10/site-packages/evaluate/loading.py", line 748, in load evaluation_module = evaluation_module_factory( File "/data/Anaconda3/envs/opencompass/lib/python3.10/site-packages/evaluate/loading.py", line 681, in evaluation_module_factory raise FileNotFoundError(

Other information

happen when evaluate truthfulqa

Modas-Li avatar Dec 28 '23 08:12 Modas-Li