ragas

ragas copied to clipboard

Evaluating LlamaIndex example hangs

5

I am new to ragas and trying to learn how to evaluate LlamaIndex. When I tried the following code, which is copy-and-pasted from https://github.com/explodinggradients/ragas/blob/main/docs/howtos/integrations/llamaindex.ipynb, the code just hangs forever after...

yjlee1011

Context Recall returning NaN when using GPT-4 models

3

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** When using any gpt-4 model as an evaluator, the context recall metric returns...

FranciscoAlves00

Answer Semantic Similarity gives output not in range 0-1

6

**Describe the bug** The output of the AnswerSimilarity metric should be in range 0-1, but gives also negative output, and not in range 0-1. This affects also answer correctness metric....

AnnamariaGalante

Faithfulness return 1 when it should not

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I have noticed that in some cases when the retrieved context is...

lalehsg

Metric to account for entities

5

**Note**: This is just an idea, open to discussions. **Problem**: For use cases(like financial, legal, health-care) where the data might contain names of people, places, organizations etc, most of the...

sky-2002

Generate Synthetic Dataset from pinecone index

3

As an RAG developer, I already have a huge data ingestion pipeline managing data ingestion for me in the Pinecone. Since ETL processes are computationally expensive, one would want to...

rnbokade

Low quality generated synthetic dataset

3

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** When I try to generate a synthetic dataset with my docs,...

WoutDeRijck

I want to know how Testset Generator to create dataset works. Can anyone give like flowchart or anything. I wanna know the high level understanding

6

I checked the [documentation](https://docs.ragas.io/en/latest/concepts/testset_generation.html) and related resources and couldn't find an answer to my question. I wanna know the high level understanding. I got to know what is evolution but...

sfc-gh-akashyap

How to interpret the combination of metrics: context precision and the rest (real world example)

9

I ran ragas to evaluate my LangChain-powered chatbot (it's basically a QA chain with document retrieval) and I got the following results. question | ground_truth | faithfulness | answer_relevancy |...

younes-io

Answer_correctness and answer_semantic_similarity do not work with Bedrock embeddings: 'BedrockEmbeddings' object has no attribute 'embed_text'. Did you mean: 'embed_query'?

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. Bedrock embeddings throw exception when used for answer_correctness and answer_semantic_similarity. See the code and error...

MikhailKozineHBS