ragas

ragas copied to clipboard

Integrating third-party LLMs for Evaluating Chinese-native RAGs

4

Hi there, Thank you for bringing the elegant RAG Assessment framework to the community. I am an AI engineer from Alibaba Cloud, and our team has been fine-tuning LLM-as-a-Judge models...

hurenjun

Manually translate prompts from English to Chinese

6

- [x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Question** Hi. I've tried language adaptation, it works totally fine the first time....

Xiyuche

Error while creating synthetic dataset

11

Hi @dosu, while using this line of code from langchain_community.document_loaders import DirectoryLoader loader = DirectoryLoader('C:\\Users\\kepa') documents = loader.load() It says directory not found error, now everytime I take care of...

knpunk

Cannot generate synthetic data using documented code, gives error

1

[X ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am following [this RAGAs documentation](https://docs.ragas.io/en/stable/getstarted/testset_generation.html) to generate test data, but I...

dividor

cannot import name 'context_precision' from 'ragas.metrics'

1

[X ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am trying to follow [this](https://docs.ragas.io/en/stable/getstarted/evaluation.html) documentation, but it doesn't work because...

dividor

Failed to parse output. Returning None.

1

[ X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** When calling evaluate on Ragas using AWS Bedrock, there are some examples...

surengunturuamazon

Can Custom Metrics be added?

6

[x] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** what is unclear to you? What would you like to know? Can...

LDelPinoNT

From [SyncLinear.com](https://synclinear.com) | [R-281](https://linear.app/exploding-gradients/issue/R-281/add-callbacks-to-trace-reasoning-of-evaluations)

jjmachan

[R-289] better support for ollama models

1

ref * [Handle large context windows using Ollama's LLMs for evaluation purpose · Issue #1120 · explodinggradients/ragas](https://github.com/explodinggradients/ragas/issues/1120) feats * check how good ollama models are performing * context window issue,...

jjmachan

[R-291] RuntimeError: There is no current event loop in thread 'MainThread'.

6

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** ``` RuntimeError: There is no current event loop in thread 'MainThread'. sys:1:...

Mervyn-1