ollama-python
ollama-python copied to clipboard
ollama
Tool calls are not properly returned when chat() is called with stream=True
I have been working on integrating Ollama tools with oterm.
When making a call to AsyncClient.chat() with tools defined and stream=True the response gives the tools to be called and params in the content attribute of the json response. For example:
{'role': 'assistant', 'content': '{"name": "get_current_location", "parameters": {}}'}
Same call with stream=False has, as it should, an empty content and the tools inside the tool_calls attribute.
I'm also experiencing this. And to add a bit more context, there is no information in the response chunks to indicate that a tool call was chosen
{'model': 'mistral', 'created_at': '2024-09-27T15:55:56.521147Z', 'message': {'role': 'assistant', 'content': '[{"'}, 'done': False}
{'model': 'mistral', 'created_at': '2024-09-27T15:55:56.551562Z', 'message': {'role': 'assistant', 'content': 'name'}, 'done': False}
{'model': 'mistral', 'created_at': '2024-09-27T15:55:56.583256Z', 'message': {'role': 'assistant', 'content': '":"'}, 'done': False}
...
Related: https://github.com/ollama/ollama/issues/5796, "Streaming for tool calls is unsupported"
