auto-rag-eval
auto-rag-eval copied to clipboard
Code repo for the ICML 2024 paper "Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation"
when running this command : python -m ExamGenerator.question_generator --task-domain StackExchange I get the following error File "C:\auto-rag-eval\auto-rag-eval\auto-rag-eval\ExamGenerator\question_generator.py", line 121, in raw_exam_generator.batch_generate_exam( File "C:\auto-rag-eval\auto-rag-eval\auto-rag-eval\ExamGenerator\question_generator.py", line 41, in batch_generate_exam data = json.load(f)...
The Data preprocessor.py is almost the same except for some comments and the object class name. Diff provided below: ```diff --- Data/StackExchange/preprocessor.py 2024-07-04 12:42:46 +++ Data/SecFilings/preprocessor.py 2024-07-04 12:42:46 @@ -12,10...
Hello! This looks really interesting. You mention a failed_questions notebook in the README for experimenting with your own LLM, but I can't find it in the repository. I hope you...