ragas icon indicating copy to clipboard operation
ragas copied to clipboard

Creating Testdataset from dictionary/json file

Open almariscal opened this issue 1 year ago • 0 comments

Describe the Feature In some cases I think it would be interesting to generate a testdataset variable from a json file or dictionary with the necessary keys (question, answer, context...) in order to also evaluate LLMs on human-generated Testdataset for specific topics.

Why is the feature important for you? Because I am implementing RAGAS on a really specific topic and some tests have to be carried out on exames generated by specialists in the company.

Additional context I would be able to generate the code and a Pull request. Maybe the feature is already implemented but I have not found any method "from_json" or similar.

almariscal avatar Oct 11 '24 16:10 almariscal