ragas

ragas copied to clipboard

[R-282] use callbacks to trace reasoning of `Prompt`

3

Hi, at first, I want to say thanks this wonderful work. Actually, I want save the reason of evaluation. I mean, line 69 to line 106 at https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/_answer_correctness.py Actually, it...

JinSeoung-Oh

[R-290] Failed to parse output. Returning None. - SimpleEvolution - TestsetGenerator

11

I am getting an “Failed to parse output. Returning None” error. I I have tried llama3 and mistral8x7b. I believe both models should be able to generate json like outputs....

JPonsa

- [X] Test generation from blogs - [X] resolve errors associated with headlines splitter - [X] Implement hierarchical splitter - [X] Experiment with persona wise generation - [ ] Add...

shahules786

[R-229] Automatic language adaptation bugs cleanup

3

- [X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** When i try automatic language adaptation for a faithfulness metric, i get...

rere950303

From [SyncLinear.com](https://synclinear.com) | [R-286](https://linear.app/exploding-gradients/issue/R-286/release-prompt-object-as-experimental)

jjmachan

Implementation of Noise sensitivity metrics from RAGChecker

12

Solves: - #1185 - Took inspiration from RAGChecker from AWS Noise sensitivity [noise sensitivity](https://github.com/amazon-science/RAGChecker/tree/main/ragchecker) metrics. - Have tested it locally, it is working giving the results. ### Input ```python from...

sahusiddharth

generate_with_langchain_docs is broken

23

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** Running `generate_with_langchain_docs` gets stuck, showing: ``` Filename and doc_id are the same for...

rolandgvc

[R-263] Roadmap - v0.2

9

 - [ ] #1010 - [ ] Reference free - [ ] Generation - [ ] Summarisation - [ ] Code summary - [ ] Textual summary - [...

jjmachan

Fixed metric adaptation issue with the prompt factory

2

Issue: Runtime metric language adaptation (e.g. adapting to italian and then to german in the same run) does not work. The reason is that the @dataclass factories in the metrics...

ssoima

[FIX] - Fix for summarization edge case

1

This PR adds a fix for the issue mentioned in #1108 However I have a points to discuss @shahules786 : - I had added `conciseness_score` to penalize long summaries, but...

sky-2002