ragas

ragas copied to clipboard

Evaluation with IBM WatsonX LLM's

1

Hey! I have tried my hands on RAGAS with Watson LLM, the major issue I am facing is getting the warning: "Failed to parse output. Returning None." Continuously. Is there...

swayam-khandelwal

About RagChecker

4

Recently, I have noticed something similar to ragas, which is the [RagChecker](https://github.com/amazon-science/RAGChecker). It provides a new perspective to evaluate RAG pipelines which separately focuses on: - Relevant chunk - Irrelevant...

binghangli378

Ensuring Consistent Question Sets for Summarization Score Evaluation

6

Hi! I have a question regarding the generation of question sets for the Summarization Score metric. I am working on creating a high-quality summary and need reliable evaluation metrics to...

noweymik

TestsetGenerator.from_langchain Generating failed, randomly stuck at 0% to 80%.

1

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** TestsetGenerator.from_langchain Generating failed, randomly stuck at 0% to 80% And I've...

pp6699

Answer Relevance doesnt need context

2

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** the answer relevance metric, does not need context as input, it doesnt...

mohamad-tohidi

Synthetic dataset generation produces undesired Introductory Phrases with Bedrock LLM

5

[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** Hi, I'm trying to use this library for generating a synthetic dataset...

CosaroLisa

Automatic language adaptater is not working

29

I have followed the steps of the "How-To-Guide" about Automatic Language Adaptation (https://docs.ragas.io/en/v0.1.9/howtos/applications/use_prompt_adaptation.html) and tried to modify the already implemented classes but I could not resolve my problem. **Describe the...

Jeerhz

Can you provide human assessment data mentioned in RAGAS paper?

2

**Describe the Feature** Can you could provide the human assessment data collected for bechmarking RAGAS metrics against human evaluations in your [paper](https://arxiv.org/pdf/2309.15217)? **Why is the feature important for you?** The...

awsvmaringa

[Question] How to change custom mode name

4

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. ** How do I specify the name of the model** An LLM is...

landhu

Hook into Runnable RAG chain

2

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** How can I integrate EvaluatorChain with an existing Runnable RAG chain?...

amitjoy