ragas

ragas copied to clipboard

question and ground_truth is always the same when generating conditional_question test case.

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** question and ground_truth is always the same when generating conditional_question test case....

francescofan

BadRequestError: Unsupported data type while using generator using testset generation

5

I have checked the https://github.com/explodinggradients/ragas/blob/main/docs/howtos/integrations/llamaindex.ipynb and related resources and couldn't resolve my bug. When I try to generate the testset from the text file using generator, it gives the following...

Jothimalar1997

Faithfulness Errors

2

- [x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** I got the following error. ERROR:ragas.executor:Exception raised in Job[9]: AttributeError('NoneType' object...

TakutoIyanagi-littletree

Message 'o statements were generated from the answer' was sent when evaluating and the result is like this: Faithfulness:nan

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Message 'o statements were generated from the answer' was sent when evaluating...

francescofan

Adding custom question and ground_truth to the Ragas Testset.

2

How can we add custom questions and ground truth to the testset generated using Ragas TestSetGenerator: ``` from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context generator = TestsetGenerator.from_llama_index(...

adityamity

AttributeError('ContextRecallClassificationAnswers' object has no attribute '__root__')

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I cannot evaluate open source gemma:2b model using ragas. I actually get...

nprime496

Is there any example using local open source LLM model for evaluation?

1

Using openAI api might not be feasible for some environment, could someone provide any reference or example link on running evaluation with local LLM model/?

RyanTree-HS

subclass of LLM

20

I wrote this code and I am using a subclass of LLM and BASERAGASEMBEDDINGS. I already have the embeddings and I face the error to set the API key.to solve...

amin-kh96

- This pull request introduces a new workflow for deploying documentation, updates versioning information, and makes minor adjustments to documentation and configuration files. The most important changes include the addition...

zodwick

The runner thread which was running the jobs raised an exeception.

8

--------------------------------------------------------------------------- ExceptionInRunner Traceback (most recent call last) Cell In[56], [line 22](vscode-notebook-cell:?execution_count=56&line=22) [11](vscode-notebook-cell:?execution_count=56&line=11) embeddings = client.embeddings.create( [12](vscode-notebook-cell:?execution_count=56&line=12) input=inputs, [13](vscode-notebook-cell:?execution_count=56&line=13) model="RAG_text-embedding-3-large" [14](vscode-notebook-cell:?execution_count=56&line=14) ) [16](vscode-notebook-cell:?execution_count=56&line=16) generator = TestsetGenerator.from_langchain( [17](vscode-notebook-cell:?execution_count=56&line=17) generator_llm, [18](vscode-notebook-cell:?execution_count=56&line=18) critic_llm, [19](vscode-notebook-cell:?execution_count=56&line=19)...

tushitdave