ragas

ragas copied to clipboard

I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. I am trying to use reasoning model(gpt-5-mini), for generating the metrics. But the langchain wrapper is seeming...

KiranPrasath-26

Getting always 0 in agent evaluation with Agent Goal accuracy

2

Hi Team, I am using ragas ==0.2.15. I have created an Investment Research Assistant in LangChain-based conversational AI system designed to guide users in making informed investment decisions. It supports...

divyamuralidharan-ai

Support for Batch API

1

**Describe the Feature** Is it possible to support Batch API for Ragas? **Why is the feature important for you?** Batch API has lower cost, so it'll lower the evaluation cost...

minhrongcon2000

Feedback Request: Future of Testset Generation Module in Ragas v0.4

2

Hello Ragas community! 👋 As we prepare for the **Ragas v0.4 release**, we'd like to gather your feedback on the future direction of our Testset Generation module. This module has...

jjmachan

problem with answer_relevance - all time NAN for local llm

1

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** I want to make ragas score for custom llm and i...

spynet-us

**Summary** Integrate UQLM’s response-level confidence scoring (bounded from 0 to 1) into Ragas via new metrics that calls UQLM `BlackBoxUQ` scorers and/or `WhiteBoxUQ` when token-level logprobs are available. These metrics...

dylanbouchard

AspectCritic not working with openai o3

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am using AspectCritic metric with custom settings of the evaluator model....

taborzbislaw

Duplicated document node generated during knowlege graph creation

5

- [ YES ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** duplicated document nodes are generated during building knowlege graph Ragas...

xizhou-vw

Check RAG chunk relevance

2

[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** I want to check if RAG chunks are relevant for the user...

jonrosner

The output answer is the same as the reference answer, and the blue score has always been 0

2

The output answer is the same as the reference answer, and the blue score has always been 0 demo： if __name__ == '__main__': query = "今天天气怎么样" reference = """今天天气很不错""" response...

LillyChen