Force Claude3 models to output JSON object and parse it more reliably

[OB42]

Anthropic models seems to struggle to output valid json objects and it was causing Skyvern to fail +/- randomly, sometimes the anthropic output looked like this:

Here's a JSON object : {...}

Which caused the function parse_api_response to fail.

We can force it to output json by prefilling the response with "{": https://docs.anthropic.com/claude/docs/prefill-claudes-response

I also changed the way JSON is extracted from markdown, because the way it was done before might be too unreliable if some of the json data contain backticks in a string.

And I removed any text before and after the first and last braces of the data, Anthropic sometimes seemed to struggle to output the extraction format and was outputting text after the object.

Warning: This was only tested using Anthropic(with Claude 3 Opus) as I do not have OpenAI credits at the moment. Feel free to test it/edit it as necessary.

[suchintan]

you're two steps ahead of my discord comment

[suchintan]

~@OB42 can we try this approach first? https://docs.anthropic.com/claude/docs/control-output-format~

[OB42]

@OB42 can we try this approach first? https://docs.anthropic.com/claude/docs/control-output-format

Well I wrote this code after reading that documentation.(the first link I sent is also linked on the documentation you sent, just below that https://docs.anthropic.com/claude/docs/control-output-format#prefilling-claudes-response )

return [{"role": "user", "content": messages}, {"role": "assistant", "content": "{"}]

The other approach seems to specify the desired format, but that seems to already be the case in the prompts(?)

[suchintan]

@OB42 You're right -- that was a silly comment by me!

[suchintan]

This is great. @ykeremy can you review?

[ykeremy]

Will merge https://github.com/Skyvern-AI/skyvern/pull/293 with additional changes, closing this PR. @OB42, thank you so much for the contribution

[OB42]

Will merge #293 with additional changes, closing this PR. @OB42, thank you so much for the contribution

You're welcome! :) sorry for not adding the required changes quicker 😅

[suchintan]

Will merge #293 with additional changes, closing this PR. @OB42, thank you so much for the contribution

You're welcome! :) sorry for not adding the required changes quicker 😅

This was a fantastic contribution! Keep em coming!