ragas
ragas copied to clipboard
explodinggradients
Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
This PR addresses issue #2044 by improving the handling of local models in Ragas. ## Changes 1. Added a new `is_local_model` flag to `RunConfig` with increased timeout (10 minutes) for...
Are there any links to refer to recommended thresholds for each metric listed below for RAG evaluation? I applied these metrics to the RAG project. These are the metrics for...
I am currently generating the testset from ragas , and i can observe the test set is skewed towards few files in the directory and the distribution is not even...
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** While using azure models (AzureOpenAIEmbeddings, AzureChatOpenAI) the evaluate() module seems to fail...
**Describe the Feature** Currently, IDs are UUIDs. it would improve idempotency of knowledge graph generation if the IDs were based on hashes of the text; that way the same text...
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **My Question** https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/#llm-based-context-precision In the docs of Context Precision without reference - it...
Description: I am trying to generate test set with ragas and I am getting connection error when invoking generator.generate_with_langchain_docs with the generator object. I am using llm proxy for the...
[ ] I have checked the [documentation](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/) and related resources and couldn't resolve my bug. **Describe the bug** I'm using `TestsetGenerator.generate_with_langchain_docs` to generate 3 questions per document from a list...
Metadata
Owner
Metadata
Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines