ChatOpenAI: bind_tools not callable after with_structured_output

[p3nnst8r]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field
from langchain.tools import StructuredTool

class ResponseModel(BaseModel):
  a_value:str = Field(description="This doesn't matter much")

def a_func(val: int):
    return True

a_tool = StructuredTool.from_function(
            func=a_func,
            name="A func",
            description="A function you will need",
        )

llm = ChatOpenAI(model="gpt-4o-mini",temperature=0)
structured_llm = llm.with_structured_output(ResponseModel)
llm_with_tools = structured_llm.bind_tools([a_tool]) <----- not available

Error Message and Stack Trace (if applicable)

'RunnableSequence' object has no attribute 'bind_tools'

Description

I am attempting to retrieved structured output in a json format (to pass via an api to a frontend), and I also require calling out to tools. I cannot figure out how to combine the two, or there is an issue with code to do so.

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 24.1.0: Thu Oct 10 21:02:27 PDT 2024; root:xnu-11215.41.3~2/RELEASE_X86_64 Python Version: 3.13.1 (main, Dec 3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)]

Package Information

langchain_core: 0.3.28 langchain: 0.3.13 langchain_community: 0.3.13 langsmith: 0.2.4 langchain_experimental: 0.3.4 langchain_openai: 0.2.14 langchain_text_splitters: 0.3.4

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.10 async-timeout: Installed. No version info available. dataclasses-json: 0.6.7 httpx: 0.27.2 httpx-sse: 0.4.0 jsonpatch: 1.33 langsmith-pyo3: Installed. No version info available. numpy: 1.26.4 openai: 1.58.1 orjson: 3.10.9 packaging: 24.1 pydantic: 2.9.2 pydantic-settings: 2.6.0 PyYAML: 6.0.2 requests: 2.32.3 requests-toolbelt: 1.0.0 SQLAlchemy: 2.0.36 tenacity: 9.0.0 tiktoken: 0.8.0 typing-extensions: 4.12.2

[keenborder786]

@p3nnst8r I don't know what you are trying to achieve, but essentially, with_structured_output returns a RunnableSequence that consists of the following two Runnables:
RunnableBinding (where ChatOpenAI is binding, and the given schema is passed as an additional parameter to tools) -> OutputParser.

So, calling bind_tools again on the RunnableSequence is causing this error.

This is not recommended, but if you want to use additional tools in the same RunnableSequence, you can do the following:

structured_llm.steps[0] = structured_llm.steps[0].bound.bind_tools([a_tool, ResponseModel])

However, I still don't understand why you want to use additional tools when with_structured_output is only used to make the LLM parse the result into a specific format. It is recommended that you initiate a different instance of the LLM with the desired tools.

[AniketSaki]

In langchain-core==0.3.28, the bind_tools fn is not implemented and all basechatmodel's with_structured_output fn are calling it. https://github.com/langchain-ai/langchain/blob/a37be6dc6531b5dc68a5ccea22227da468e5ca8a/libs/core/langchain_core/language_models/chat_models.py#L1115

[kapis]

keenborder786

@p3nnst8r I don't know what you are trying to achieve, but essentially, with_structured_output returns a RunnableSequence that consists of the following two Runnables: RunnableBinding (where ChatOpenAI is binding, and the given schema is passed as an additional parameter to tools) -> OutputParser.

So, calling bind_tools again on the RunnableSequence is causing this error.

This is not recommended, but if you want to use additional tools in the same RunnableSequence, you can do the following:

structured_llm.steps[0] = structured_llm.steps[0].bound.bind_tools([a_tool, ResponseModel]) However, I still don't understand why you want to use additional tools when with_structured_output is only used to make the LLM parse the result into a specific format. It is recommended that you initiate a different instance of the LLM with the desired tools.

Not the original author, but I have the same problem.

Structured outputs and tools serve fundamentally different purpose:

1. Tools are for the LLM to trigger external functions. Tools require a feedback passed back to the LLM as a separate, LLM-native message type.
2. Structured outputs are just formats for LLM outputs. The default output format is just "text". Structured output gives a capability to customize LLM output to be a whatever JSON object. No feedback is required.

Imagine a ChatGPT-like app where LLM messages are not markdown text, but rich UI elements, implemented via with_structured_output. And this app also needs LLM to have tools - with .bind_tools. Such a basic app is impossible to implement with the current Langchain framework. Your suggestion to It is recommended that you initiate a different instance of the LLM with the desired tools is irrelevant for this app, because the app will never know if a reaction to a given user message needs to use a tool or structured output - this should be decided by the native LLM abilities.

I understand why it is like this - because just 2 years ago, when Langchain was launched, there were no LLMs that could do real Structured Output - and Langchain framework was doing the heavy-load of validating the LLM responses. But now, with OpenAI structured outputs guaranteeing correct format for every generation, it's not longer required to validate the outputs of a structured output. Understanding that this may not be true for other LLMs, I think Langchain framework should give this freedom to the developers.

[heberuriegas]

Has anyone been able to solve this with streaming response?

[kapis]

@heberuriegas I found this hack to be working with streaming:

llm_with_tools = llm.bind_tools(
        tools,
        strict=True,
        response_format={
            "type": "json_schema",
            "json_schema": {
                "name": "<name of your response schema>",
                "strict": True,
                "schema": ResponsePydanticSchema.model_json_schema(),
            },
        },
    )

The downside is you have to manually parse response content.

The hack works because langchain adds all extra kwargs provided in bind_tools to the LLM sdk client.

[Rituraj491]

@kapis I'm getting this error when I implemented your solution

TypeError: this._client.chat.completions.create(...)._thenUnwrap is not a function

[varevshatyan]

Same issue, bumping for visibility, all other threads with this same problem have been closed and the "hacky" solutions don't work reliably.

[svallory]

I'm baffled that we can't do this. Please guys, prioritize this. It is such a common scenario!

[svallory]

I needed a solution ASAP, so I came up with this (for OpenAI and Gemini) and thought I would share...

UPDATE: Since version 0.3.12, ChatOpenAI supports passing a tools kwarg to .with_structured_output so all you need to do is:

openai_llm.with_structured_output(
        schema=schema,
        method="json_schema",
        tools=tools,
        strict=True,
    )

But, in case you need to support both OpenAI and Gemini, here's the updated code:

def _bind_tools_with_structured_output(
    llm: ChatOpenAI | ChatGoogleGenerativeAI,
    schema: type[BaseModel],
    tools: list[BaseTool] = [],
) -> Runnable:
    """
    Combines the functionality of bind_tools and with_structured_output for LangChain models.

    This helper works around the limitation that bind_tools and with_structured_output
    cannot be chained together since both return Runnables.

    Args:
        llm: The base chat model (ChatOpenAI or ChatGoogleGenerativeAI)
        schema: The Pydantic BaseModel class for structured output
        tools: List of tools to bind to the model

    Returns:
        A Runnable that has both tools bound and structured output configured

    Raises:
        ValueError: If the LLM type is not supported
    """
    if isinstance(llm, BaseChatOpenAI):
        return llm.with_structured_output(
            schema=schema,
            method="json_schema",
            tools=tools,
            strict=True,
        )

    elif isinstance(llm, ChatGoogleGenerativeAI):
        model_supports_tool_choice = lambda model: (
            "gemini-1.5-pro" in model
            or "gemini-1.5-flash" in model
            or "gemini-2" in model
        )
        
        # For OpenAI, we can use tool_choice to force calling the structured output tool
        # But not all Gemini models support tool_choice
        tool_choice = (
            schema.__name__ if model_supports_tool_choice(llm.model) else None
        )

        # Add a parser to extract the arguments of the schema tool call into an instance of the schema.
        parser = PydanticToolsParser(tools=[schema], first_tool_only=True)

        try:
            # Bind all tools (user tools + schema tool) and set tool_choice to force schema output.
            # Include ls_structured_output_format for LangSmith if supported.
            bound_llm = llm.bind_tools(
                tools=list(tools) + [schema],
                tool_choice=tool_choice,
                ls_structured_output_format={
                    "kwargs": {"method": "function_calling"},
                    "schema": convert_to_openai_tool(schema),
                },
            )
        except Exception as e:
            print(
                f"Gemini bind_tools with ls_structured_output_format failed with {type(e).__name__}: {e}. Falling back."
            )
            # Fallback without ls_structured_output_format
            bound_llm = llm.bind_tools(
                tools=list(tools) + [schema],
                tool_choice=tool_choice,
            )

        return bound_llm | parser

    raise ValueError(
        f"Unsupported LLM type: {type(llm)}. Only ChatOpenAI and ChatGoogleGenerativeAI are supported."
    )

[techmangreat]

"UPDATE: Since version 0.3.12, ChatOpenAI supports passing a tools kwarg to .with_structured_output so all you need to do is:"

Please add documentation on how to use "with_structured_output". There is no documentation mentioning a "tools" argument. If the "tools" argument is used, the response is null on version 0.3.12 of langchain.

Please don't make changes without documenting them.

Also, how have people been using langchain for years, if this basic functionality (having tool calls and structured output, i.e., two basic building blocks of any AI agent) has only been implemented 3 days ago? Since this feature is experimental, please refer to alternatives, indicating how have people been using langchain for years to build AI agents.

To be able to call functions, and to be able to have structured output, are two fundamental building blocks of AI agents. How did any person build any AI agent using langchain up until 3 days ago? What solutions are there to build basic AI agents, without using experimental features from 3 days ago?

Don't tell me to refer to the documentation, as the documentation on how to build AI agents with langchain is entirely absent, and all the existing methods to build AI agents with langchain have been DEPRECATED in favor of the complete mess that is langgraph.

People just want a basic python interface to interact with AI models. Tool calls, chat history, and structured output is all we need. When will langchain support that?

[tsmith-perchgroup]

Has there been any update to this issue? Am I making some mistake?

I agree with all of the above, it seems like a very basic and fundamental scenario: having your LLM simultaneously emit tool calls and structured output.

Below is my toy code to try and get this working:

from langchain.tools import tool
from pydantic import BaseModel, Field
from typing import Literal, Optional
from models import gpt5

class AddOneSchema(BaseModel):
    n: int = Field(..., description="Number to add one to.")

class ExtractTextSchema(BaseModel):
    text: str = Field(..., description="Text to extract substring from.")
    i: int = Field(..., description="Initial index.")
    j: int = Field(..., description="Final index.")

@tool(args_schema=AddOneSchema)
def add_one(n):
    """add one to number"""
    return n + 1

@tool(args_schema=ExtractTextSchema)
def extract_text(text, i, j):
    """Extract a substring from text between two integers"""
    return text[i:j]

class NextAction(BaseModel):
    next_tool: Literal["extract_text", "add_one"]
    reason: Optional[str] = Field(..., description="Why this tool was chosen")

llm = gpt5.with_structured_output(
        schema=NextAction,
        method="json_schema",
        tools=[add_one, extract_text],
        strict=True,
)

response = llm.invoke('Use your tools to add one to two')

My response is None. What mistake am I making here?

I'm using langchain 0.3.27. gpt5 here is just an AzureChatOpenAI object, instantiated in another module.

[ChinaNuke]

I have the same result as @tsmith-perchgroup , the LLM returned {'parsed': None}.

My response is None. What mistake am I making here?

[tsmith-perchgroup]

Ok, so I'm now running langgraph==1.0.2, langchain==1.0.3 and langchain-openai==1.0.1. The below works:

llm = ChatOpenAI(...)

class ResponseFormat(BaseModel):
    output_for_user: str = Field(
        ...,
        description=(
             "Blah Blah"
        )
    )
    some_other_field: int = Field(
        ...,
        description=(
             "Blah Blah"
        )
    )

response = llm.bind_tools(
    [
        list_dir,
        update_files,
        write_to_file,
        send_email
    ],
    response_format=ResponseFormat
).invoke(messages)

print(response.additional_kwargs['parsed'])

And your parsed pydantic model will be in the additional_kwargs['parsed'] item in the response object! Hope that helps.