ragas

ragas copied to clipboard

[R-240] (docs): Document how to evaluating with a locally hosted LLM to help choose the ones that work best

9

[X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** I have a locally hosted LLM which I am intending to use as...

Exploding-squid

AssertionError: Adapted output keys set(output.keys())=set() do not match with the original output keys: output_keys[i]={'statements'}

2

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** AssertionError: Adapted output keys set(output.keys())=set() do not match with the original output...

Ailce8862

Automatic language adaption TestSet generation error: "Adapted output keys do not match with the original output keys"

9

**Describe the bug** Can't get the automatic language adaption going for testset generation. I retried this about 10 times. Ragas version: 0.1.4 Python version: 3.11 **Code to Reproduce** Share code...

baswenneker

Adapted output keys set(output.keys())={'深度', '相关性', '清晰度', '结构'} do not match with the original output keys: output_keys[i]={'structure', 'clarity', 'depth', 'relevance'}

3

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** ``` >>> generator.adapt(language, evolutions=[simple]) Traceback (most recent call last): File "", line...

qism

[R-230] Adapt prompts "in place" also for cached prompts

2

**Describe the Feature** Adapt Prompts "in place", also in case they are retrieved from the cache. **Why is the feature important for you?** If you want to generate a testset...

HerrIvan

Generate a Synthetic Test Set In Other Language

3

I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question. **Your Question** what is unclear to you? What would you like to know? when I...

AprilJoy

sentence_segmenter in metric should be adapt to language in adapt function?

2

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** It's good that almost all metric in ragas can be adapt to...

jmgu0212

Buggy tutorials

2

[ X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug. **Describe the bug** A clear and concise description of what the bug is. This tutorial...

amr-cloudforce

Keep getting "Your authentication token is not from a valid issuer", although I got OpenAI access

8

**Describe the bug** I'm using AzureChatOpenAI. I was able to create a model instance, to send a prompt and to get an answer: ``` chat_model = AzureChatOpenAI( base_url="https://abc.verycoolcorp.com/etx-bot/openai/deployments" + "/gpt-35-turbo",...

ilaychen

Reference free * Generation * Summarisation * Code summary * Textual summary With Reference * Generation for data types * Text * answer correctness Code SQL From [SyncLinear.com](https://synclinear.com) | [R-264](https://linear.app/exploding-gradients/issue/R-264/add-more-metrics-with-v02)

jjmachan