ragas

ragas copied to clipboard

Failed to parse output.

1

**Describe the bug** Local LLMs either raise Timeout error or Fails to parse output. Ragas version: 0.1.15 Python version: 3.11.3 **Code to Reproduce** ```python from transformers import AutoTokenizer tokenizer =...

g-hano

What is the purpose of returned context when using synthetic data generation?

5

[X] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** what is unclear to you? What would you like to know? When...

diegodebrito

Evaluating: 100%|██████████| 50/50 [12:00<00:00, 14.40s/it] eeror encountered whil evaluating using ChatOllama

12

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** A clear and concise description of what the bug is. Ragas version:0.1.20...

plaban1981

Traditional NLP Metrics examples in docs are broken

1

[X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** The examples from the '[Traditional NLP Metrics](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/traditional/)' don't work on the current version....

dschuler36

RAGAS without ground_truth

9

Hi, is it possible to use the library without having the ground_truth for any metrics?

dasilvaviviane

RuntimeError: Task <Task pending name='Task-61' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\

4

python3.9.19 ; ragas 0.1.21 code: ```py from langchain_community.document_loaders import DirectoryLoader from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context from langchain_openai import ChatOpenAI, OpenAIEmbeddings from langchain.embeddings import HuggingFaceEmbeddings loader...

Z-oo883

Ragas testset generation throws error "TypeError: AsyncMessages.create() got an unexpected keyword argument 'n'" with LlamaIndex and Anthropic

2

[x ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am using Ragas to generate a test dataset, and using Llama-index...

ahgraber

pydantic.v1.error_wrappers.ValidationError: 2 validation errors for DataRow

3

code: ```py from langchain_community.document_loaders import DirectoryLoader from ragas import run_config from ragas.executor import Executor from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context from langchain_openai import ChatOpenAI, OpenAIEmbeddings from...

Z-oo883

How to adapt language of result just exactly the same to the contexts given which is separated into several files and the files are in different languages.

23

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** As the document of ragas says in "Automatic language adaptation"(https://docs.ragas.io/en/v0.1.1/howtos/applications/use_prompt_adaptation.html#language-adaptation-for-metrics)，we can...

francescofan

metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

1

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** when i import library from ragas i get metaclass conflict: the...

vohoangkh4ng