ragas

ragas copied to clipboard

## Summary This PR adds a base_url parameter to the embedding_factory function, allowing users to specify a custom API endpoint for the embedding model. This is particularly useful when working...

KoLogicDev

Calling generate 3 times gives the questions with exact words every time. generate_multiple tends to give semantically matching questions.

mshobana

Exception raised in Job[2]: RagasOutputParserException(The output parser failed to parse the output including retries.)

5

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** what is unclear to you? What would you like to know?...

shanchao0906

Getting scores with llamaindex using custom model as bedrock or gemini

3

**Describe the Feature** I would like to do evaluation using Bedrock or Gemini with llamaindex and not just langchain (https://docs.ragas.io/en/stable/howtos/customizations/customize_models/). i have already tried to use to use `LlamaIndexLLMWrapper` and...

mdciri

The description of Context Relevance metric was not right and it was not giving the explanation which it actual does. Updating it to the right one.

mshobana

# Description Closes #1990 by adding type checks and improved error handling. This issue was a bit unintuitive when I came across it, and is easily fixed by having a...

ziggycross

FactualCorrectness() failed to fix bad intermediate output

2

Sometimes, FactualCorrectness() produces incorrect intermediate output. Example: "claims":[ "the number is 6", "the number is 9", ] Then, it tries to fix the error. However, even though it regenerates the...

Nyoko74

Seperate Evaluation for Retrieval and Generation both

3

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** How ragas can make it possible to evaluate retrieval and generation...

PrashantDixit0

ragas evaluator function is taking LLM model as none even it is passed in evaluator's function call

1

Code: %python # %pip install openai langchain from ragas.metrics import answer_relevancy, faithfulness, context_utilization from ragas.evaluation import evaluate from azure.identity import DefaultAzureCredential import openai import os from langchain.llms import AzureOpenAI #...

manaligc

Nvidia Accuracy calculation code is not consistent with the docs

4

[ x ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** In the Nvidia Accuracy metric calculation, the User Answer and Reference...

vitorsabbagh