langserve icon indicating copy to clipboard operation
langserve copied to clipboard

Published 20 hours ago • verified publisher icon langchain-ai

include_callback_events=True with add_routes() not working with agentExecutor

Open giraudmartin opened this issue 1 year ago • 2 comments
trafficstars

Hello 👋

I have trying to add a callback to my AgentExecutor with langserve but I got some errors. I have an already implemented route in my backend so I've added the parameter include_callback_events=True to this route :

class Input(BaseModel):
    input: str
    chat_history: List[Union[HumanMessage, AIMessage, FunctionMessage]]

class Output(BaseModel):
    output: Any


# Create chat model from Azure AI
chatModel = AzureChatOpenAI(
   openai_api_version="2023-07-01-preview",
   azure_deployment="gpt-35-turbo_1106",
)

# Create agent with create_openai_tools_agent
tools = [myTool]
agent = create_openai_tools_agent(chatModel, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)

add_routes(
    app,
    agent_executor.with_types(input_type=Input, output_type=Output),
    path="/backend",
    include_callback_events=True
)

I have a React client that calls this route thanks to the RemoteRunnable import :

import { RemoteRunnable } from "langchain/runnables/remote";

const result: { output: string } = await remoteChain.invoke({
  input: question,
  chat_history: chatHistory
}, { callbacks: [langfuseHandler] });

And I have the following errors from langserve, (note that it happens also even without callback from client) :

Traceback (most recent call last):
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 74, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/langserve/server.py", line 464, in invoke
    return await api_handler.invoke(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/langserve/api_handler.py", line 719, in invoke
    self._InvokeResponse(
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 27 validation errors for backendInvokeResponse
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_start' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_start',))
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_end' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_error' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_error',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> messages
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chat_model_start' (type=value_error.const; given=on_agent_finish; permitted=('on_chat_model_start',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> prompts
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_llm_start' (type=value_error.const; given=on_agent_finish; permitted=('on_llm_start',))
callback_events -> 15 -> __root__ -> response
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_llm_end' (type=value_error.const; given=on_agent_finish; permitted=('on_llm_end',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> input_str
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_start' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_start',))
callback_events -> 15 -> __root__ -> output
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_end' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_error' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_error',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> query
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_start' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_start',))
callback_events -> 15 -> __root__ -> documents
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_end' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_error' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_error',))

It seems quite related to this ticket, but it doesn't have any fix : https://github.com/langchain-ai/langserve/issues/283

Thanks a lot for your help !

giraudmartin avatar Mar 15 '24 11:03 giraudmartin

  1. Apologies don't use callbacks yet, callback code is still untested so I don't have confidence in it working.
  2. For now work-around if you need token counting is to use astream_events and count tokens client side with tik-token. I'll address this at some point, but token counting is not high priority right now

eyurtsev avatar Mar 15 '24 13:03 eyurtsev

@eyurtsev Ok noted, thanks for your return 👍

giraudmartin avatar Mar 15 '24 15:03 giraudmartin