NeMo-Guardrails
NeMo-Guardrails copied to clipboard
The provider Ollama does not support streaming.
I tried referring to this guide and use Ollama as the engine but got this message. How do i enable streaming for Ollama-based models?
Hi @kohjx1
You are getting this error (as mentioned in the guide) because the Ollama model does not natively support token-by-token streaming. You can refer to this link from LangChain for an in depth overview. Let me know if this helps.