Bug: trim_messages and ChatAnthropic token counter with tools

[ccurme]

Privileged issue

• [x] I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.

Issue Content

ChatAnthropic.get_num_tokens_from_messages uses Anthropic's token counting API and will error for invalid message sequences. These constraints are not respected inside trim_messages. See below for examples.

from functools import partial

from langchain_anthropic import ChatAnthropic
from langchain_core.messages import AIMessage, HumanMessage, ToolMessage
from langchain_core.messages.utils import trim_messages
from langchain_core.tools import tool


@tool
def get_weather(location: str):
    """Get the weather."""
    pass

tools = [get_weather]


llm = ChatAnthropic(model="claude-3-5-sonnet-latest").bind_tools(tools)


messages = [
    HumanMessage("hi"),
    AIMessage("hello"),
    HumanMessage("what's the weather in florida?"),
    AIMessage(
        [
            {"type": "text", "text": "let's check the weahter in florida"},
            {
                "type": "tool_use",
                "id": "abc123",
                "name": "get_weather",
                "input": {"location": "florida"},
            },
        ],
        tool_calls=[
            {
                "name": "get_weather",
                "args": {"location": "florida"},
                "id": "abc123",
                "type": "tool_call",
            },
        ],
    ),
    ToolMessage(
        "It's sunny.",
        name="get_weather",
        tool_call_id="abc123",
    ),
]

Example 1

trim_messages(
    messages,
    max_tokens=200,
    token_counter=llm,
)

Breaks when we attempt to count tokens in a reversed list of messages: https://github.com/langchain-ai/langchain/blob/db8201d4dafb533133cc51c87c7ef011e546e03f/libs/core/langchain_core/messages/utils.py#L1309-L1312

BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.0: tool_result block(s) provided when previous message does not contain any tool_use blocks'}}

Example 2

trim_messages(
    messages,
    max_tokens=200,
    token_counter=llm,
    strategy="first",
)

Breaks because ChatAnthropic.get_num_tokens_from_messages expects tools to be passed in as a kwarg: https://github.com/langchain-ai/langchain/blob/db8201d4dafb533133cc51c87c7ef011e546e03f/libs/partners/anthropic/langchain_anthropic/chat_models.py#L1160-L1162

BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Requests which include tool_use or tool_result blocks must define tools.'}}

Example 3

trim_messages(
    messages,
    max_tokens=200,
    token_counter=partial(llm.get_num_tokens_from_messages, tools=tools),
    strategy="first",
)

Breaks because an invalid history is sent to the token counting API:

BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.3: Messages containing tool_use blocks must be followed by a user message with tool_result blocks'}}