Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
Example Code
class PictureCategorizationOutputState(BaseModel):
"""Picture categorization output state model.
Attributes:
picture_type: The picture type.
image_texts: The image texts.
operational_error: The operational error, if you encounter any issue during your task.
"""
picture_type: AIPictureType = Field(description="The type of the picture.")
image_texts: list[ImageText] = Field([], description="The texts found in the image.")
dimensions: Dimensions = Field(description="The dimensions of the image in pixels (height and width).")
operational_error: str = Field(
"", description="The operational error, if you encounter any issue during your task."
)
highlighted_image_b64: str = Field(description="The b64 encoded image with highlighted text.")
def question_vehicle_picture_category_node(state: PictureCategorizationSharedState) -> PictureCategorizationOutputState:
"""Prompt AI for vehicle picture type (INTERIOR / EXTERIOR).
Args:
state: Picture categorization shared state.
Returns:
AIVehiclePicture: Vehicle picture
"""
llm = ChatGoogleGenerativeAI(
model="gemini-1.5-pro",
temperature=0,
project="some_project"
).with_structured_output(PictureCategorizationOutputState) #!!!!!! THIS CRASHES !!!!!!!!!
prompt = get_picture_categorization_prompt_template(str(state.picture_url))
return cast(PictureCategorizationOutputState, (prompt | llm).invoke({}))
Error Message and Stack Trace (if applicable)
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2024.2.4\plugins\python-ce\helpers\pydev\pydevd.py", line 1570, in exec
pydev_imports.execfile(file, globals, locals) # execute the script
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\JetBrains\PyCharm 2024.2.4\plugins\python-ce\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:\dev\lizy-ai\projects\picture_categorization\graph_v1.py", line 49, in
compiled_graph.invoke({"picture_url": "https://media.gq.com/photos/6508829d305ef4e0229049b3/master/pass/plane.jpg"})
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\pregel_init.py", line 1929, in invoke
for chunk in self.stream(
^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\pregel_init_.py", line 1649, in stream
for _ in runner.tick(
^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\pregel\runner.py", line 105, in tick
run_with_retry(t, retry_policy, writer=writer)
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\pregel\retry.py", line 44, in run_with_retry
task.proc.invoke(task.input, config)
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\utils\runnable.py", line 410, in invoke
input = context.run(step.invoke, input, config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langgraph\utils\runnable.py", line 184, in invoke
ret = context.run(self.func, input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\lizy-ai\projects\picture_categorization\nodes.py", line 28, in question_vehicle_picture_category_node
).with_structured_output(PictureCategorizationOutputState)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langchain_google_genai\chat_models.py", line 1239, in with_structured_output
tool_choice = _get_tool_name(schema) if self._supports_tool_choice else None
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langchain_google_genai\chat_models.py", line 1383, in _get_tool_name
genai_tool = tool_to_dict(convert_to_genai_function_declarations([tool]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langchain_google_genai_function_utils.py", line 173, in convert_to_genai_function_declarations
fd = _format_to_gapic_function_declaration(tool)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langchain_google_genai_function_utils.py", line 197, in _format_to_gapic_function_declaration
return _convert_pydantic_to_genai_function(tool)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\langchain_google_genai_function_utils.py", line 270, in _convert_pydantic_to_genai_function
function_declaration = gapic.FunctionDeclaration(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\message.py", line 728, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\marshal.py", line 235, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\rules\message.py", line 45, in to_proto
return self._wrapper(value)._pb
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\message.py", line 728, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\marshal.py", line 233, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\marshal.py", line 235, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\rules\message.py", line 45, in to_proto
return self._wrapper(value)._pb
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\message.py", line 728, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\marshal.py", line 235, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\marshal\rules\message.py", line 45, in to_proto
return self._wrapper(value)._pb
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\giloz\AppData\Local\pypoetry\Cache\virtualenvs\lizy-ai-8ngQPScW-py3.12\Lib\site-packages\proto\message.py", line 724, in init
raise ValueError(
ValueError: Unknown field for Schema: title
Description
it seems the "Title" property is giving an issue during the marshalling. same basic structure is working with ChatOpenAI class.
System Info
System Information
OS: Windows
OS Version: 10.0.22631
Python Version: 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]
Package Information
langchain_core: 0.3.21
langchain: 0.3.9
langchain_community: 0.3.9
langsmith: 0.1.147
langchain_anthropic: 0.2.4
langchain_chroma: 0.1.4
langchain_fireworks: 0.2.5
langchain_google_genai: 2.0.6
langchain_google_vertexai: 2.0.8
langchain_openai: 0.2.11
langchain_pinecone: 0.2.0
langchain_text_splitters: 0.3.2
langgraph_sdk: 0.1.43
Optional packages not installed
langserve
Other Dependencies
aiohttp: 3.9.5
anthropic: 0.40.0
anthropic[vertexai]: Installed. No version info available.
async-timeout: Installed. No version info available.
chromadb: 0.5.21
dataclasses-json: 0.6.7
defusedxml: 0.7.1
fastapi: 0.115.6
filetype: 1.2.0
fireworks-ai: 0.15.9
google-cloud-aiplatform: 1.74.0
google-cloud-storage: 2.19.0
google-generativeai: 0.8.3
httpx: 0.27.2
httpx-sse: 0.4.0
jsonpatch: 1.33
langchain-mistralai: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
numpy: 1.26.4
openai: 1.57.0
orjson: 3.10.12
packaging: 24.2
pinecone-client: 5.0.1
pydantic: 2.9.2
pydantic-settings: 2.6.1
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.36
tenacity: 8.2.3
tiktoken: 0.8.0
typing-extensions: 4.12.2
langchain copied to clipboard
langchain-ai
