ragas

ragas copied to clipboard

[R-279] [R-280] Summarization Score Formula is unreasonable

7

I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I read the doc for summarization Score. https://docs.ragas.io/en/stable/concepts/metrics/summarization_score.html  QA score, **higher is better**....

ofooo

ModuleNotFoundError: No module named 'ragas.langchain'

17

**Description of the bug** Using Google Colab. After running `!pip install ragas`, unable to import RagasEvaluatorChain from ragas.langchain.evalchain. It was okay last week (v0.0.22). Ragas version: 0.1.0 Python version: 3.10.12...

xinyang-handalindah

Fix ensure_ascii for json.dumps

1

Hi! This PR simply fixes all `json.dumps`. With `ensure_ascii=False`, unicodes can be properly serialized into JSON files in the modern days when UTF-8 is basically universal. This may also prevent...

ifsheldon

Invalid n value (currently only n = 1 is supported)

11

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** ``` openai.BadRequestError: Error code: 400 - {'detail': 'Invalid n value (currently only...

jidechao

[R-247] Integrations: wandb and wandb tracer

5

todo - [ ] How to do Evaluation Driven Development with the wandb stack? - [ ] build out `ragas.integration.wandb` to make that easiest for users From [SyncLinear.com](https://synclinear.com) | [R-247](https://linear.app/exploding-gradients/issue/R-247/integrations-wandb-and-wandb-tracer)

jjmachan

adapt pydantic.v1.error_wrappers.ValidationError: 1 validation error for Prompt

6

I want to customize the adapt function, change the language to Chinese this is code： ```py cache_dir = ".cache" adapt( metrics=evaluate_stk, language="Chinese", cache_dir=cache_dir, llm=llm ) ``` and this is the...

runwean

Custom prompt for test set generation

19

Is there a way to customize a prompt using the TestsetGenerator class? I looked at the documentation, and it only provides examples with default prompts used under the hood. I...

harvey1992

ValueError: a cannot be empty unless no samples are taken

1

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. ValueError: a cannot be empty unless no samples are taken **Describe the bug** A clear...

Qin-xb

Answer Relevancy giving same questions everytime

7

@dosu I've gone through the library and first of all, the prompt needs to have something that instructs to go through the context, rather than just providing in few shot....

knpunk

[R-293] Noise sensitivity metrics from RAGChecker

2

RAGChecker from AWS has [noise sensitivity](https://github.com/amazon-science/RAGChecker/tree/main/ragchecker ) metrics that could be useful to Ragas users TBD 1) Implement metrics 2) Prepare docs to intuitively explain how it's calculated and how...

shahules786