griptape
griptape copied to clipboard
OllamaPromptDriver doesn't stream responses when using tools
In Griptape 1.3.0, the issue with OllamaPromptDriver not working with Tools when stream was enabled was resolved, but now the responses aren't streaming at all.
See this example:
from griptape.drivers import OllamaPromptDriver
from griptape.structures import Agent
from griptape.tools import DateTimeTool
from griptape.utils import Stream
driver = OllamaPromptDriver(model="llama3.2", stream=True)
agent = Agent(prompt_driver=driver, tools=[DateTimeTool()])
for artifact in Stream(agent).run("What's today's date? Write a song about it"):
print(artifact.value, end="", flush=True)
If you run this with the tools the response won't stream - but if you disable the tool it does.
@shhlife unfortunately this is blocked on the ollama side. See: https://github.com/ollama/ollama/issues/5796 https://github.com/ollama/ollama/issues/7886