[bug] Let json validation pass when a None-able field is missing from the LLM response

[zsimjee]

Describe the bug JSON validation fails when a field that is Optional or Unioned with None is not present. In those cases, JSON validation should pass instead

To Reproduce

import openai
from guardrails import Guard
from pydantic import BaseModel, Field
from typing import Optional

class Pet(BaseModel):
    name: str = Field(description="name of pet")
    age: Optional[int] = Field(description="Do not add this to the response")

guard = Guard.from_pydantic(output_class=Pet)

# ensure json parser doens't get confused
raw_llm_output, validated_output, *others = guard.parse("""{
    "name": "fido"     
}""")

print(validated_output)


raw_llm_output, validated_output, *others =  response = guard(
    llm_api=openai.chat.completions.create,
    prompt="""Generate a pet object. Only add fields described below. Do not add any new fields
    
    
    ${gr.complete_json_suffix}"""
)

print(validated_output)
print(raw_llm_output)
print(others)

Expected behavior We expect this to pass validation instead of failing it if a valid 'name' comes back but an age does not

Library version: Version 0.3.x

Additional context Related to this thread on discord

[CalebCourier]

Related to this: In general our conversion of Unions from a Pydantic model to our internal DataType is lacking. Right now it expects every union to be discriminated based on a sub-property. However there are use cases that call for a basic OR union without a discriminator applied. This might be the root of the issue here since an Optional is just a Union with None, but I can't confirm that.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 14 days with no activity.