ragas

ragas copied to clipboard

Local LLM with Ragas evaluation issue

6

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I am trying to use a local LLM in the evaluate function.,...

SalwaMostafa

[R-243] docs: Add documentation on the best approach to define custom metrics.

3

[X] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. I was exploring the possibility to create custom metrics. It seems that it is...

ouphi

Vunerable version for the onnx package in the poetry.lock

2

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I’ve identified that the onnx package listed in the [src/experimental/poetry.lock](https://github.com/explodinggradients/ragas/blob/ee6ff5f3eb7540ef3fb35cfbc17bd3e1c1d0e7cb/src/experimental/poetry.lock#L598) file has...

linqiu0-0

Swiftide integration

3

Hey everyone, amazing work on Ragas! Recently I've been working on [Swiftide](https://github.com/bosun-ai/swiftide), and I'm looking to integrate with an evaluation framework. Rust is great at performance, Python is king at...

timonv

Why Context Precision and Context Recall are yielding a binary score instead a range between 0 and 1?

8

Anjalisoni99-byte

ValueError: a cannot be empty unless no samples are taken

8

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** ValueError: a cannot be empty unless no samples are taken Ragas version:...

Rugved2204

Getting Error: Runner in Executor raised an exception in Ragas evaluate using Ollama and giving Nan value in df

4

I encountered an issue while evaluating a dataset using the ragas library with the Langchain LLM and Sentence Transformer embeddings. The process throws an exception during execution. **Steps to Reproduce:**...

divrajput

RAGAS with huggingface models

8

**Describe the bug** A clear and concise description of what the bug is. I tried using RAGAS with a model that is not OpenAI. In general whatever model I use...

SalvatoreRa

Getting error "Failed to parse output. Returning None" on faithfulness metric

8

I am getting error "Failed to parse output. Returning None" on faithfulness metric for some inputs. This is inconsistent behavior as it is haphazard and sometimes works, sometimes doesn't for...

ableiweiss

Problem with `answer_relevancy` Metric in Ragas Framework

6

### GitHub Issue Title: Problem with `answer_relevancy` Metric in Ragas Framework ### Description: Hello, I am experiencing an issue with the Ragas evaluation framework when using the `answer_relevancy` metric. Other...

huangxuyh