ragas

ragas copied to clipboard

How can I calculate total tokens consumption for this evaluation process, in Azure Open AI.

1

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** How to calculate total tokens consumption for evaluating our RAG model?...

wanjeakshay

Not able to calculate noise_sensitivity_relevant using Azure Open AI.

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am able to calculate 4 metrics {'faithfulness': 1.0000, 'answer_relevancy': 0.9880, 'context_recall':...

wanjeakshay

Faithfullness returns always Nan with a local LLM

18

[ x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Faithfullness is not working returning always Nan, sometimes also the context precision...

cpolcino

How to save one piece of data in a single testset generating step?

1

Hi, I am wondering how to save each generated testset data in a single testset generating step? I mean for example if I use `generator.generate_with_langchain_docs` and set `test_size=1000`, it will...

Dongchenghang

shahules786

fix: correct double swap of parameters in AnswerAccuracy metric templ…

6

## Overview This PR addresses the double swap bug in the AnswerAccuracy metric implementation. In the original implementation, both the meta-labels (User Answer / Reference Answer) and their corresponding values...

DmitrySirakov

fix: update required columns in RefusalRate (docs)

3

## Overview This PR fixes a bug in the RefusalRate docs metric implementation. The ValidationError occurred because the required fields for a SingleTurnSample were set to {"response", "reference"}, while the...

DmitrySirakov

` scores = ragas_evaluate( dataset=eval_dataset, metrics=[metric], run_config=my_run_config, llm=evaluator_llm, embeddings=evaluator_embeddings ) ` this code is normal in local windows, but when I run in ubuntu(18.04.6), it is error:Can't patch loop of...

hznnnnnn

TestsetGenerator in local Ollama

1

[ ] I have checked the [documentation](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/) and related resources and couldn't resolve my bug. **Describe the bug** I am not able to do testset generation using the local Ollama...

hanifa2102

When we generate the Test set for evaluating RAG, how to add the source file name from which the QnA is generated.

1

- I checked the [documentation](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/#load-documents) and related resources and couldn't find an answer to my question. **Question** - Previously there was a feature which would let you add the name...

zaid1212