griptape icon indicating copy to clipboard operation
griptape copied to clipboard

OllamaPromptDriver doesn't stream responses when using tools

Open shhlife opened this issue 9 months ago • 1 comments

In Griptape 1.3.0, the issue with OllamaPromptDriver not working with Tools when stream was enabled was resolved, but now the responses aren't streaming at all.

See this example:

from griptape.drivers import OllamaPromptDriver
from griptape.structures import Agent
from griptape.tools import DateTimeTool
from griptape.utils import Stream

driver = OllamaPromptDriver(model="llama3.2", stream=True)
agent = Agent(prompt_driver=driver, tools=[DateTimeTool()])

for artifact in Stream(agent).run("What's today's date? Write a song about it"):
    print(artifact.value, end="", flush=True)

If you run this with the tools the response won't stream - but if you disable the tool it does.

shhlife avatar Feb 10 '25 02:02 shhlife

@shhlife unfortunately this is blocked on the ollama side. See: https://github.com/ollama/ollama/issues/5796 https://github.com/ollama/ollama/issues/7886

collindutter avatar Feb 10 '25 18:02 collindutter