Support for GPT-4 Structured Outputs

[slavashvets]

Description

As I can understand, currently, BAML sends JSON schemas as part of the LLM prompt to structure outputs. With the recent introduction of GPT-4's Structured Outputs feature, we have an opportunity to improve our implementation by leveraging the new API.

Background

OpenAI has introduced Structured Outputs for GPT-4, which allows model outputs to reliably adhere to developer-supplied JSON Schemas. This feature is designed to ensure that model-generated outputs exactly match the provided schemas, improving reliability and reducing the need for extensive error handling and retries.

Proposed Changes

1. Implement support for the new response_format parameter in the OpenAI API calls.
2. Update the BAML language to allow users to define output schemas that can be directly used with the Structured Outputs feature.
3. Modify the code generation process to utilize the new API feature when appropriate.
4. Update documentation to reflect the new capabilities and usage.

Benefits

• Improved reliability of structured outputs
• Reduced need for error handling and retries
• Potential performance improvements by offloading schema enforcement to the API

Implementation Details

• The new feature uses a json_schema option in the response_format parameter.
• It's currently available on gpt-4o-mini and gpt-4o-2024-08-06 models.
• The implementation will need to handle the refusal field in API responses for cases where the model refuses to generate a response due to safety concerns.

[hellovai]

Thanks for the thought out request! We're actively working on this (along with function calling support). You should be able to see docs shortly for what this will look like. (In about 1 week)

[Jonovono]

@slavashvets if you are using Typescript, I made a library for making it easier to use OpenAIs structured outputs. Not nearly as nice as BAML, but might be fun to play with for now ;p. Can check it out here: https://github.com/cigs-tech/cigs. Short video using it here: https://x.com/Jonovono/status/1828549105540648993

[zeionara]

Hey! I haven't found the feature in docs, is it supported now?

[sxlijin]

Sorry for the radio silence, y'all! We come back to this every few months but have never managed to nail the abstractions we want to provide. Please do continue to let us know if you want this!

[david-storycraft]

Claude provides this now as of Nov 14th.