ragas

ragas copied to clipboard

[x ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** It is a "documentation bug". In the notebook [Non-English Testset Generation Notebook](https://github.com/explodinggradients/ragas/blob/main/docs/howtos/customizations/testgenerator/_language_adaptation.md),...

benqua

feature!: knowledge graph flexibility

1

The current implementations of the synthesizers (in particular, `MultiHopAbstractQuerySynthesizer` and `MultiHopSpecificQuerySynthesizer`) have hard-coded values for the node- and relationship- properties/types they look for when generating scenarios and samples. This works...

ahgraber

[Feautre Reuqest] [Bug] Async + RAGAS

6

**Describe the Feature** As far as I can tell, it seems like there is a lack of async functions in ragas, which leads to a lot of nesting async issues...

logan-markewich

I've added the [PEP 561](https://peps.python.org/pep-0561/) `py.typed` marker file to support type checking. Without it, mypy is raising the following error on imports from `ragas`: ``` Skipping analyzing "ragas": module is...

sisp

Hi! I would like to know if RAGAS metrics are as reliable as with traditional metrics, especially compared to retrieval metrics such as precision@k, recall@k, MRR, nDCG... Are there any...

iosuAbal

Context Precision score heavily influenced by position of relevant contexts in array

5

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** ContextPrecision evaluator in RAGAS produces inconsistent scores that are strongly influenced by...

mshobana1990

when use ragas0.2, unable to apply transformation: Connection error

5

when I try to use ragas0.2, I have encountered a problem：unable to apply transformation: Connection error，what shoud I do to deal with it? **enviroment:** ragas=0.2,python=3.9 **code**: ```py from langchain.document_loaders import...

Z-oo883

I want to see integration of ragas with crewAI or autogen for Agent evaluation like tool evaluation or agent goal

aggarwaln1

[ERROR] Prompt xxx failed to parse output:The output parser failed to parse the output including retries.

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I have the same problem. When the length of reference and response...

674717172

Prompt ${xx_prompt} failed to parse output: The output parser failed to parse the output including retries.

3

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** what is unclear to you? What would you like to know?...

hsx1612727380