ragas

ragas copied to clipboard

Why did the Kernal crashing issue get removed?

2

Why did the Kernal crashing issue get removed? Was this fixed? The issue wasn't changed to close but just disappeared.

davidzimmerman19

Mistral-8x7b answer_correctness metric prompt fix

1

[X ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** Is there any prompt modification such that Mistral-8x7b does not give...

AlexisGitHu

Are the Context Precision Examples and Calculation Correct?

3

[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** When calculating context precision, don't you need to provide each sentence/statement in...

pmbaumgartner

Possible bug in evaluate function

3

**Describe the bug** I have been using the evaluate function in ragas to obtain different metrics of a RAG system, the evaluation consists on 51 questions with gorund truth. I...

abetatos

Evaluation Dataset - Ground truth Warnings

1

Hi, I would like to raise a potential error around the dataset used in the evaluation method (from `evaluation.py`). I got the evaluation method working when I set the following...

lsantosq

Missing components

4

**Describe the bug** The latest commit has deleted a bunch of components and code references, this does not seem intentional / is not documented anywhere. Ragas version: commit/bad3a8e1d115e69bb4acbd9887ca5b6cb6a03fba Python version:...

nils-hde

Support positively grading 'I don't know' answers when the context is not relevant

2

**Describe the Feature** Hi there, I read all metric descriptions and realized that there is no support for the following case: Suppose the LLM is asked a question, and there...

mayalinetsky-kryon

Add tests for `llm.json_load`

4

**Describe the Feature** bugs like the one @pberger514 caught in #593 might occur down the road too. We need tests for these sections of code. The catch here is that...

jjmachan

Generate a Synthetic Test Set Program has issue

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I have used the code to generate Synthetic Test set for my...

Kunal56377

Answer relevancy returns either zero or average cosine similarity

1

[ x ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** I've observed that the current methodology for calculating answer relevancy...

JJLZ21