ragas

ragas copied to clipboard

`evaluate` supporting replicates

1

**Describe the Feature** It would be nice to do something like `evaluate(..., num_replicates=30)` so I can calculate mean/std dev of accuracy on a benchmark `EvaluationDataset`. What I mean by replicates...

jamesbraza

Remove hard context lenght limitation on 7000 words on metrics `nv_relevance` and `nv_groundedness`. Updated Judge x HumanCorrelation Leaderboard.

titericz

Evaluation Errors: Output Parsing and Timeout Issues for Context/Faithfulness Metrics

6

[√ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** I'm encountering issues when trying to evaluate my RAG system using...

WangAo-0

How can we integrate Ragas with WebdriverIO

3

Could you please advise how we can integrate Ragas with WebdriverIO

hameeda2

[ yes] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** template_relevance1 is used twice but template_relevance2 is not used. I think...

zjt148680

for `Context Precision without reference` definition, i.e LLMContextPrecisionWithoutReference metric can be used when you have both retrieved contexts and also reference contexts associated with a user_input. Here, it should be...

AbhishekRP2002

**Introduction:** This PR addresses a issue related to outdated initialization in the Ragas Getting Started documentation. The existing example code uses the deprecated RAG() class, resulting in runtime errors when...

Vinisha-projects

Hi, This is from the website: https://docs.ragas.io/en/stable/getstarted/rag_eval/ The line: # Initialize RAG instance rag = RAG() is not working.

amscosta

[ x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** The code given in [Single Hop Query Testset documentation](https://docs.ragas.io/en/latest/howtos/applications/singlehop_testset_gen/#query-generation-using-synthesizers) does not work....

codechanger0

This PR addresses the issues reported in #2044 where local LLMs like Ollama with qwen2.5:14b were encountering RagasOutputParserException and TimeoutError during evaluation. ### Changes 1. **Increased default timeout in RunConfig...

jjmachan