Pommel4711

@PhilipMay Hey, I'm running this on Windows. Do you use Linux, or do you know how I can fix this problem? I came across this Stack Overflow post that might...

I coppied the `dataset` code [from this url](https://github.com/huggingface/datasets/blob/689447f8c86f777829a4db9ccc5d8133c12ec84c/src/datasets/load.py#L113-L134) ans now i get this error ```bash (lighteval) D:\Arbeit\AIUI\RAG Telecom Dataset\lighteval>python run_evals_accelerate.py ^ --model_args "pretrained=DiscoResearch/DiscoLM_German_7b_v1" ^ --tasks "./examples/tasks/all_german_rag_evals.txt" ^ --override_batch_size 1 ^...

> Hi @Pommel4711 , Did you try to update `datasets` first as @lhoestq suggested? Yes, I did update datasets as @lhoestq suggested. > OSes that don't support SIGALRM are supported...

# Issue with `lighteval` Evaluation Script ## Description I completely removed the Conda environment `lighteval` and updated the repository using the following command: ```sh git pull git checkout main ```...

I tried running `pip install -U datasets` to upgrade `datasets` as you suggested, instead of manually editing the files. Unfortunately, this error still persists. ```sh ImportError: cannot import name 'metric_module_factory'...

## Maybe i found the problem with the dataset. I followed the steps mentioned in this [comment](https://github.com/huggingface/lighteval/issues/211#issuecomment-2222375717) to resolve the issue without deleting the file `C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\datasets\load.py` and replacing it with...

> Are you trying to run evaluation in offline mode? I got the same error but I am trying offline and I have replace HF links with local location but...