ragas

ragas copied to clipboard

Open Analytics

10

We track very basic usage metrics to guide us to figure out what our users want, what is working and what's not. As a young startup, we have to be...

jjmachan

Cannot generate TestDataset more than once

6

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Calling TestsetGenerator.generate_with_langchain_docs() more than once results in documents from the first call being...

njcolvin

OpenAI Tokens Usage for RAGAs Evaluation

5

I am using RAGAs evaluation metrics to evaluate my RAG pipeline. Is there any way I could know the openai tokens used per api call from RAGAs, lets say for...

aliai22

Ragas Synthetic Test Data Generation error using AzureOpenaiEmbeddings

5

I was trying to do "test generation" using RAGAS framework with the help of the "https://docs.ragas.io/en/stable/concepts/testset_generation.html", I'm facing error. Please have a look on the below error. ragas.exceptions.ExceptionInRunner: The runner...

sona-16

From [SyncLinear.com](https://synclinear.com) | [R-255](https://linear.app/exploding-gradients/issue/R-255/remove-async=true-from-ragas-evaluate)

jjmachan

Grammar and punctuation improvements in critique prompts

1

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I've noticed a few grammar and punctuation issues in the [critique prompts](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/critique.py). The...

ruankie

[R-224] Ragas integration with Langfuse to trace both llm outputs and scores in the same place

1

[x ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. I checked and tested the integration of `ragas` with `Langfuse` as demonstrated [here](https://docs.ragas.io/en/latest/howtos/integrations/langfuse.html)....

databill86

[R-246] Could it be a mistake in the prompts?

1

Hello, I assume that there is a minor mistake in find_relevant_context_prompt under the testset/prompt.py. relevant contexts seem not right to me. Is it possible to be? The first one is...

cemsoyleyici

Feature suggestion - Handling LLM quotas when evaluating

7

It would be nice to handle LLM quotas when evaluating a large dataset, for my personal case I cannot increase the default 60 request per min for VertexAI LLM. Tracking...

0ENZO

Customizing Testset Generation: list index out of range

7

**Your Question** - When customising Testset Generation, I used `InMemoryDocumentStore` and `LCDocument` from `langchain`. - An error occurred during calling `generate_with_langchain_docs` function, during debugging, the detailed error message is as...

GolfHotelSierra