ragas

ragas copied to clipboard

faithfulness_score: nan

5

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** faithfulness_score: always be nan **Code Examples** from agent import * from...

beatG123

ValueError: Can't patch loop of type <class 'uvloop.Loop'>

5

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am trying to integrate RAGAS with an API based on FastAPI, and...

gustavo-has-stone

I am getting error while synthetic test data generation

6

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am getting error while synthetic test data generation using hugging face...

wanjeakshay

Automatic language adaptation Bugs. Cannot do with german language

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** A clear and concise description of what the bug is. Ragas version:...

DatTNH23

RAGAS TimeoutError Exception while Evaluation

2

I have a self-created data set with questions, GT, contexts and answers and have started the evaluation with the RAGAS evaluate() method. The percentage increases a few points and then...

kaan9700

## Experimental feature: Component in metrics ```python # load desired NLI model as component from ragas.experimental.metrics.component import TextClassificationNLIComponent classifier = pipeline( "text-classification", model='vectara/hallucination_evaluation_model', tokenizer=AutoTokenizer.from_pretrained('google/flan-t5-base'), trust_remote_code=True ) classifier = classifier(input_pairs, top_k=1)...

shahules786

division by zero when computing output score

2

Good time of the day, Here https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/filters.py#L60 is a division by zero, which I encounter when using ragas with ollama+llama3-8b. To solve it, I can replace: ``` output["score"] = sum(output.values())...

theoden8

Exception raised in Job[2]: ValueError(Got unknown type content=

6

#My code is from datasets import Dataset from ragas.llms import LangchainLLMWrapper from langchain_community.embeddings import SparkLLMTextEmbeddings, HuggingFaceEmbeddings #from langchain_community.chat_models import ChatSparkLLM from sparkai.llm.llm import ChatSparkLLM SPARKAI_URL = 'wss://spark-api.xf-yun.com/v3.5/chat' SPARKAI_APP_ID = ''...

tombabycome

AnswerCorrectness raises the AssertionError: "AssertionError: AnswerSimilarity must be set"

5

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. Hi experts, I use the answer_correctness as the metric, but it failed due to **AssertionError:...

xidian237

feat: add faithfulness metric based on Bespoke Labs MiniCheck model

7

This PR adds a faithfulness metric based on the [Bespoke-MiniCheck-7B](https://huggingface.co/bespokelabs/Bespoke-MiniCheck-7B) model. Users can use the metric either by calling the model through the Bespoke Labs API, or by running the...

vutrung96