Enforcing json format

[Leahh02]

• [x] I checked the documentation and related resources and couldn't find an answer to my question.

My Question I'm getting exception

 File "/my/path/to/ragas/prompt/pydantic_prompt.py", line 423, in parse_output_string
    raise RagasOutputParserException()
ragas.exceptions.RagasOutputParserException: The output parser failed to parse the output including retries.

And I'm pretty sure it's happening because the LLM is getting cut off when it's generating the answer in the JSON format. Right now I have the max tokens for my LLM to generate set to 1024. I can increase the max tokens to help avoid that problem. But I'm wondering what happens if the LLM has a longer generation and still runs into that max token count before it finishes the JSON structure. I know the prompt says to follow the JSON schema, so there's an effort to make sure that doesn't happen, but could there be a more concrete way for RAGAS to make sure that JSON format is finished? Or is that just too complicated?

I might also decrease my temperature and tweak my top_p to decrease the likelihood of that over generation. Maybe there could be a reminder/guidance on settings like that somewhere in the documentation. That might give people some trouble shooting ideas and a reminder that tweaking those settings might give them better performance.

Code Examples After line 201 in pydantic_prompt.py I'm printing out the output string to see what the LLM generates. That's where I saw that the JSON schema wasn't getting finished. Here's a shortened example of what it looks like:

 {
    "statements": [
        {
            "statement": "The crash of Yeti Airlines Flight 691 in Pokhara, Nepal was caused by a technical issue with the plane.",
            "reason": "The context explicitly states that the crash was caused by the aircraft's propellers being feathered, which is a technical issue.",
            "verdict": 1
        },
        {
            "statement": "The weather was clear at the time of the crash.",
            "reason": "There is no"