evaluate
evaluate copied to clipboard
huggingface
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
For some reason when i run this: ``` print(evaluate.list_evaluation_modules(include_community=False)) ``` I get an empty list. ``` evaluate==0.4.2 ```
Not sure if this feature belongs to this library or would it require a complete separate library. I am proposing the creation of a library where llm benchmarks can be...
When relying on `sklearn`, the help message that should be printed is `pip install scikit-learn`, not `pip install sklearn`.
 i am running evaluate==0.4.2 on python 3.11 and i get this error, please help.
I am trying to compte f1 score ,My predictions are mostly numbers whreas my references are sometimes malayalm sting version of these numbers. Thus I am getting a value error...
for sklearn, if needing install it, the ImportError should write "pip install scikit-learn" instead of "pip install sklearn" now: File "/home/ubuntu/wubing/evaluate/src/evaluate/loading.py", line 265, in _download_additional_modules raise ImportError( ImportError: To be...
When calling .compute in distributed multi-node setting, I get this error - ``` [rank1]: File "/ext3/miniconda3/envs/venv/lib/python3.8/site-packages/transformers/trainer.py", line 2750, in _evaluate [rank1]: metrics = self.evaluate(ignore_keys=ignore_keys_for_eval) [rank1]: File "/ext3/miniconda3/envs/venv/lib/python3.8/site-packages/transformers/trainer_seq2seq.py", line 180, in...
The underlying sklearn function exposes the `zero_division` attribute [1], which is not a keyword argument in the metric wrapper in this library. This means, users get a warning when there...
Hi folks, Thank you for maintaining this excellent package! I've just updated mauve's readme based on the following: - the pip package outputs two new fields: `mauve_star` and `frontier_integral_star` -...
Metadata
Owner
Metadata
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.