ragas

ragas copied to clipboard

Without this I get exceptions due to right quotes not being encoded correctly.

aubford

HOW to use LOCAL LLMS and Embeddings for ragas？

8

I have down model weights in my computer，but i don't how to use LOCAL LLMS and Embeddings for ragas according to Here is My code but it did't work ```...

minglong-huang

unable to apply transformation: 'headlines' property not found in this node

1

**Your Question** What should I do when I encountered these errors when generating test data in a non English language? **Code Examples** ```from langchain_community.document_loaders import DirectoryLoader, TextLoader from ragas.llms import...

Z-oo883

Can I generate multi-hop reference_contexts Only? (without the question and reference)

2

**Your Question** I was wondering if the API support only generating the reference_contexts and not the whole flow? In addition, where can I find the actual links the context is...

GraderYuval

ValidationError: 1 validation error for SingleTurnSample reference Input should be a valid string [type=string_type, input_value=['恐龙名字来源于...意为恐怖的蜥蜴'], input_type=list] For further information visit https://errors.pydantic.dev/2.9/v/string_type

3

This is the data I wrote with ChatOpenAI as the interface of tongyi, but I encountered the following problems, why? import os from langchain_openai import ChatOpenAI, OpenAIEmbeddings from datasets import...

729973389

Testset(samples=[])

1

I executed the following code, but the generated dataset is empty. What could be the issue? My Ragas version is 0.2.14. code{from ragas.testset import TestsetGenerator generator = TestsetGenerator(llm=generator_llm, embedding_model=generator_embeddings) dataset...

txygjx

How to evaluate rags in other language

3

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** How to evaluate rags in other language. 1. The dataset has...

landhu

FactualCorrectness false negative calculation

3

**Describe the bug** The calculation of answer_correctness metric (0.75*factual_correctness+0.25*semantic_similarity), does not use the same implementation of factual_correctness as when FactualCorrectness is computed directly. This discrepancy leads to inconsistent results in...

alvaroalfaro612

When calculate factual correctness because of fuzzy matching, the following could happen: 4 claims in response 2 claims in ground truth Let's say 2 claims in response are good, and...

tinodj

error when apply_transformers

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** A clear and concise description of what the bug is. apply_transforms never...

godisboy0