giskard

giskard copied to clipboard

According to this issue, some RAGAS metrics are not properly computed by RAGAS (this includes context recall, context precision and faithfulness).

To fix it:

• added a retrieved_documents argument to the evaluate method
• allowed the answer_fn to return retrieved documents alongside the answer to a question

GSK-3513 Fix RAGAS support metric

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: giskard/rag/evaluate.py

Function	Unhandled Issue
evaluate	ValueError: Must provide either the api_version argument or the OPENAI_API_VERSION environment variable ... Event Count: 1
evaluate	TypeError: string indices must be integers, not 'str' main in model_pr... Event Count: 1
evaluate	NameError: name 'doc_chat' is not defined _main... Event Count: 1
evaluate	AttributeError: 'EvaluateRAG' object has no attribute 'custom_model' src.evalua... Event Count: 1

---

_{Did you find this useful? React with a 👍 or 👎}

@pierlj looks good, can you add a test on the ragas metrics to make sure they are calculated correctly?

Good for me, @pierlj do you want to make a last check?

Yep, I will have a look

Please retry analysis of this Pull-Request directly on SonarCloud