JsonSchemaValidator: inconsistency in corrective message generation for Claude vs. OpenAI models

[vblagoje]

Describe the bug

The JsonSchemaValidator class, designed to validate JSON content of ChatMessage against a specified JSON Schema, exhibits inconsistent behavior in generating corrective messages when used with Claude compared to OpenAI models. While the validator successfully prompts OpenAI models to generate messages that satisfy the schema requirements, it fails to elicit the correct behavior from Claude models. This inconsistency affects the utility of Haystack 2.x recovery loops, where the JsonSchemaValidator plays a crucial role in ensuring that LLM responses adhere to predefined schema constraints.

Expected behavior

The expected behavior is for the JsonSchemaValidator to work properly across different LLMs, including but not limited to OpenAI and Claude models. Upon encountering a validation error, the validator should be able to generate an error message that instructs any LLM to produce a response that correctly aligns with the specified JSON Schema. This ensures that the recovery loop mechanism can effectively guide LLMs towards generating schema-compliant responses, regardless of the underlying model architecture or provider.