json_schema / Describe / LLM structured output support

[lukepighetti]

• Javascript (zod) and Python (pydantic) LLM apis use their data class libraries to describe structured outputs for LLMs. example
• dart_mappable has full awareness of the field structure during runtime which is required for this example
• there is a protocol for structured output (it's json_schema) (see details)

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-2024-08-06",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful math tutor. Guide the user through the solution step by step."
      },
      {
        "role": "user",
        "content": "how can I solve 8x + 7 = -23"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "math_response",
        "schema": {
          "type": "object",
          "properties": {
            "steps": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "explanation": { "type": "string" },
                  "output": { "type": "string" }
                },
                "required": ["explanation", "output"],
                "additionalProperties": false
              }
            },
            "final_answer": { "type": "string" }
          },
          "required": ["steps", "final_answer"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
  }'

TODO

• [ ] add a "description" key to field annotations
• [ ] add a Map<String, dynamic> jsonSchema getter on ClassMapperBase

Why we should care

this would make dart the third language to have data classes that drive LLM responses as a first class feature

[Ehesp]

@schultek would love this too - we've got some upcoming plans which this would fit perfectly with

p.s. just sponsored you monthly for your work on this + other ecosystem packages 🥇

[lukepighetti]

good idea! just donated one-time as a thanks for dart_mappable and in the hopes that it will help with this feature request. thanks for all that you do!

[lukepighetti]

other reasons to care: this would be a step towards driving UI (like forms) from a data class

[schultek]

Hi, thanks for the sponsorship to both of you!

This seems like a pretty straightforward thing to add. I'm currently on vacation and could only work on it when I'm back. If you want to open a PR instead I'm happy to give some pointers.

[Kypsis]

This feature would be most welcome indeed.

[dkbast]

The new GenUI seems to be using this already, would be interesting to know how they are integrating that with their data classes: https://pub.dev/packages/json_schema_builder ( https://github.com/flutter/genui/tree/main/packages/json_schema_builder )