ragas

ragas copied to clipboard

TestsetGenerator -> RuntimeError: ... got Future <..> attached to a differen t loop

3

This is related to: https://github.com/explodinggradients/ragas/issues/681 **Describe the bug** TestGeneration launches an error with default example of https://docs.ragas.io/en/stable/concepts/testset_generation.html with VertexAI models. Ragas version: 0.1.7 Python version: Python 3.10.13 **Code to Reproduce**...

abetatos

Testset Generation. RateLimitError

5

**Describe the bug** Rate limit reached for gpt-3.5-turbo-0125 on tokens per min (TPM): Limit 80000, Used 79767, Requested 736. Please try again in 377ms. Visit https://platform.openai.com/account/rate-limits to learn more. Ragas...

Sansan4ez

Invalid response format. Expected a list of dictionaries with keys 'verdict'

15

Ragas version: 0.1.1 Python version: 3.8 ### **Describe the bug** Getting all NaN scores. I am using open-source LLM and hosting in vLLM openAI server style, using HuggingfaceEmbeddings and creating...

ajaychinni

answer relevancy metric OpenAI API key not found!

8

**Describe the bug** OpenAI API Key is not found by answer relevancy metric, but works for all other metrics? Environment variable "OPENAI_API_KEY" is set using dotenv. Ragas version: 0.0.19 Python...

chank20

I'm not able to reproduce documentation example

6

**Describe the bug** Hello, I'm new using this library and the first that I tried is to replicate [the few examples from the documentation.](https://docs.ragas.io/en/stable/getstarted/testset_generation.html) The example explains that the final...

bluetab-roger-p

Issue with "Bring Your Own LLMs"

12

**Describe the bug** In your tutorial, you say to use the openai_api_key "no-key", but then in your code, you throw and error if "no-key" is used as the API key....

NigelNelson

error: RagasEvaluatorChain with non-openai models

16

**Describe the bug** Cannot get the evaluation with Langchain QA Chains using non-openai models. Ragas version: 0.0.21 Python version: 3.10.0 **Code to Reproduce** ``` # list of metrics we're going...

arm-diaz

[VertexAI] ChatVertexAI object has no attribute 'set_run_config' and `'PromptValue' has no len()'

8

**Describe the bug** I am trying to use RAGAS with VertexAI (The integration with OpenAI is working fine). When calling the `evaluate()` method, I am getting: `ragas/metrics/base.py", line 116, in...

sebastian-piedoux-dv

Getting nan(not a number) when evaluating faithfulness

8

Hi, I'm trying to evulate faithfulness by giving some out of context question and I was getting the score as nan(Not a number). I took a look at the code...

aldrinjenson

Testset generator not working using AzureOpenai key.

10

I am trying to generate synthetic data using azure openai api, taking long time to run and after that getting error. Ragas version: 0.1.1 Python version: 3.10 **Code to Reproduce**...

rahul1-995