Proposal: Abstracted JSON Schema Generator for AI Model Integration

[nallwhy]

While working with both OpenAI and Gemini models using JSON responses, I noticed that the way each API handles json_schema differs quite significantly. Currently, the json_schema details are provided externally (outside of the LangChain library), but to seamlessly support multiple AI providers without requiring the user to handle these differences manually, it would be valuable to introduce an abstraction layer for defining schemas.

I’d like to propose the addition of: • An abstracted, unified representation of json_schema. • AI-specific schema generators that can transform this abstract schema into the correct format for each provider (e.g., OpenAI, Gemini).

With such an abstraction in place, it would open the door to utilities that could easily convert existing structures like Typespecs, Ecto Schemas, or Ash Resources into this unified schema format. This would significantly reduce the friction when integrating structured data responses across different AI models.

Would love to hear thoughts on this idea! Happy to contribute if there’s interest.

[nallwhy]

Currently, ash_ai(https://github.com/ash-project/ash_ai) uses the OpenAPI specification provided by ash_json_api. However, I feel that this approach may not be expressive enough for our needs. For example, Gemini introduces features like propertyOrdering which are outside the scope of the standard OpenAPI spec.

[nallwhy]

This repo provides useful references on how to write JSON schemas for each LLM.

https://github.com/samchon/openapi/blob/master/src/structures/IChatGptSchema.ts

[nallwhy]

@brainlid

Following the discussion here, I ended up creating lang_schema. It’s a library that defines an abstract schema format and provides converters to generate provider-specific JSON schemas (e.g., OpenAI, Gemini).

The goal is to minimize manual changes when switching between different AI providers by offering a unified schema representation that can be adapted as needed.

[brainlid]

Hi @nallwhy! Is the idea to write in plain JSON Schema and use the converter to adapt it? Or do you have something else in mind?

From your tests, that's what it looks like you're doing.

If that's the case, then I'm open to merging in your work to LangChain itself. The LangChain.FunctionParam turns into standard json schema, then your work could convert it if needed based on the model or config or something. The FunctionParam's are intended for the simple examples that cover all of my needs.

I'd love to hear your thoughts!

[nallwhy]

Currently, lang_schema takes a slightly different approach. Instead of adjusting the JSON schema converted from FunctionParam for each AI provider, it introduces an intermediary step using the concept of an "abstract schema". This approach is intentional, as JSON schema itself often already contains properties that might not align well with the requirements of various AI providers(e.g. additionalProperties).

Specifically, lang_schema abstracts away some of the lower-level details and focuses on a more generalized, provider-agnostic format. The idea is to go through a pipeline like FunctionParam -> abstract schema -> AI provider-specific JSON schema, instead of FunctionParam -> JSON schema -> AI provider-specific JSON schema.

To support this, I’ve added an adapter for FunctionParam to lang_schema. You can see a concrete example in this test case: https://github.com/nallwhy/lang_schema/blob/main/test/lang_schema/adapter/langchain_function_param_test.exs

This approach can be integrated with functions like get_parameters/1 in LangChain.ChatModels.ChatOpenAI, making it easier to support multiple AI providers without requiring significant changes to the core schema definitions.

Would love to hear your thoughts on this approach!

[nallwhy]

Alternatively, users can directly define the parameters_schema in a Function without going through FunctionParam, using lang_schema for more flexibility.

For this reason, I believe it’s better for lang_schema to be used independently when creating parameters_schema for Function or json_schema for chat models, rather than integrating it deeply throughout the entire LangChain codebase. Overly tight integration might actually make LangChain more cumbersome for users.

So, if this approach seems reasonable, perhaps we could position lang_schema as a recommended but optional tool for schema generation, allowing LangChain users to adopt it where it makes the most sense.