auto-rag-eval icon indicating copy to clipboard operation
auto-rag-eval copied to clipboard

Code repo for the ICML 2024 paper "Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation"

Results 3 auto-rag-eval issues
Sort by recently updated
recently updated
newest added
trafficstars

when running this command : python -m ExamGenerator.question_generator --task-domain StackExchange I get the following error File "C:\auto-rag-eval\auto-rag-eval\auto-rag-eval\ExamGenerator\question_generator.py", line 121, in raw_exam_generator.batch_generate_exam( File "C:\auto-rag-eval\auto-rag-eval\auto-rag-eval\ExamGenerator\question_generator.py", line 41, in batch_generate_exam data = json.load(f)...

The Data preprocessor.py is almost the same except for some comments and the object class name. Diff provided below: ```diff --- Data/StackExchange/preprocessor.py 2024-07-04 12:42:46 +++ Data/SecFilings/preprocessor.py 2024-07-04 12:42:46 @@ -12,10...

Hello! This looks really interesting. You mention a failed_questions notebook in the README for experimenting with your own LLM, but I can't find it in the repository. I hope you...