ragas

ragas copied to clipboard

There are a lot more LLM providers for whom we don't have parsers. We would really apprecite you inputs here in 1. suggesting parsers that you need. This will help...

jjmachan

Empty generation does not raise a `ragas.exceptions.ExceptionInRunner`

1

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Hi! I'm currently working with `ragas` to test different RAG architectures, so I'm...

Gwenn-LR

Addresses https://github.com/explodinggradients/ragas/issues/886. Taking into account solution made by this [PR](https://github.com/explodinggradients/ragas/pull/887) but also checks if temperature was set under langchain object. This is critical because on TGI endpoints no generation is...

fealbuqu

Do we need to chunk documents before text set generation?

3

The embedding model is used for TestsetGenerator: ```py generator = TestsetGenerator.from_langchain(generator_llm, critic_llm, embedding_model) dataset = generator.generate_with_langchain_docs(documents, test_size=100, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25}) ``` Does this mean we have to...

hanfei1986

get keyphrases from TestGenerator

2

Is it get the keyphrases extracted by TestGenerator using the API?

ableiweiss

Highly programmable knowledge graph-based synthetic data generation grounded on documents. PR raised #1024 From [SyncLinear.com](https://synclinear.com) | [R-270](https://linear.app/exploding-gradients/issue/R-270/synthetic-test-data-generation)

jjmachan

ValidationError with OpenAIEmbeddings in Ragas

2

A `ValidationError` occurs when trying to use the `evaluate` function from Ragas with the `OpenAIEmbeddings` model. The error message suggests using `AzureOpenAIEmbeddings` when using Azure, but I am using the...

SushmitaSingh96

- [ ] make ragas metrics deployable as a server - [ ] make testset generation interactive with an API making a simple documentation site to showcase how this might...

jjmachan

Detailed Proposal: [PromptObject - v0.2](https://ragas.notion.site/PromptObject-v0-2-e7bbe8cc7fd94ac8868e51d6493520f1) ## User Requirements 1. Create Simple prompts easily 2. Typed I/O 3. Adaption - stress test, use the current issues to understand the limitations 4....

jjmachan

RAGAS Framework into CI/CD Model for RAG Pipeline

5

[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** How can we use RAGAS within a CI/CD model for a RAG...

maraevfd