[Enhancement]: Handle non-string outputs gracefully in auto_contains_json evaluator

[aybruhm]

Description

This PR enhances the auto_contains_json evaluator to handle cases where the output is not a string more gracefully, ensuring that any untracked errors lead to a clear and informative failure.

Related Issue

Closes AGE-573

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Aug 23, 2024 2:06pm
agenta-documentation	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Aug 23, 2024 2:06pm

[aybruhm]

Thanks @aybruhm, quick question 👇 Shouldn't this apply to all non-RAG evaluators (not just contains_json) ?

We already are doing that. The contains_json evaluator requires that the value of data is a str-JSON, and not another type. See comment here.

[jp-agenta]

QA'd in oss-local. QA in cloud-staging pending.