ragas

ragas copied to clipboard

Random RuntimeError: Tool context error detected. This can occur due to parallelization in VertexAI

5

- [x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I run evaluation, sometimes it works sometimes it fails with : `RuntimeError:...

franck-cussac

The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead ( For Ragas Version 0.1.10)

3

**Description:** Hi, I am currently using the latest version of Ragas (version 0.1.10) and encountering an error when using Ragas with local LLMs from Ollama. Here is my code: ```python...

mb16biswas

From [SyncLinear.com](https://synclinear.com) | [R-276](https://linear.app/exploding-gradients/issue/R-276/langchain-langsmith-integration)

jjmachan

todos - [ ] documentation for langsmith and langchain integrations - [ ] the execute() method gets stuck in unexpected places

jjmachan

Invalid n value (currently only n = 1 is supported

2

File "/usr/local/lib/python3.10/dist-packages/ragas/langchain/evalchain.py", line 166, in evaluate dataset_with_scores = self.metric.score(dataset, callbacks=callbacks) File "/usr/local/lib/python3.10/dist-packages/ragas/metrics/base.py", line 76, in score score = self._score_batch(dataset.select(batch), callbacks=group) File "/usr/local/lib/python3.10/dist-packages/ragas/metrics/_answer_relevance.py", line 123, in _score_batch results = self.llm.generate( File...

ARES3366

Hello，calculate faithfulness Error: No statements were generated from the answer

5

[ ]Hello I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Hello, when I run a sample like this with chinese: from datasets...

13416157913

answer_correctness giving inconsistent result

1

**Describe the bug** The answer correctness does not seems consistent. **Code to Reproduce** question = ["1","2","3","4","5"] answers = ["yes", "no", "no", "yes", "no"] contexts = [["1","2","3","4"],["1","2","3","4"],["1","2","3","4"],["1","2","3","4"],["1","2","3","4"]] ground_truth = ["yes", "no","yes","no",...

chboudry

Prompt for Question Creation or Am I missing something

1

**Question:** Where is the prompt generating the actual question. I am using Claude Sonnet for generating the question test data set and it seems like the prompt can use some...

TheShoes

Q: Aspect Critique: Multiple Verdicts?

1

[X] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. https://docs.ragas.io/en/stable/concepts/metrics/critique.html **Your Question** In the documentation (linked above), there is a calculation reference to...

dkhundley

Synthetic Dataset Generation fail due to "ValueError: 'a' cannot be empty unless no samples are taken"

1

Hi, I am trying to use `TestsetGenerator` to produce a synthetic dataset paired with `LlamaIndex` and 'Ollama', it successfully completes the embedding process, but before startin the generation process the...

antoninoLorenzo