Evaluation by RAGAs

[psmukhopadhyay]

Hello

Is it possible to link PrivateGPT with RAGAs metrics (https://docs.ragas.io/en/stable/concepts/metrics/index.html#ragas-metrics) for evaluation? Can you please guide me how to achieve this?

Best

[jaluma]

Glad to heard about evaluation. You can start looking at llamaindex integration: https://docs.ragas.io/en/stable/howtos/integrations/llamaindex.html

If you check the doc, you will see that you can already use llamaindex on datasets to evaluate them. For LLM, you can inject LLMComponent and get the PGPT model. For the testing part of the retriever, you will have to externalise or give access to the retriever along with all the node_postprocessor.