langserve icon indicating copy to clipboard operation
langserve copied to clipboard

Streaming RemoteRunnable not working w/ rag-chroma-private

Open rlancemartin opened this issue 2 years ago • 2 comments

Testing rag-chroma-private.

for -

from langserve.client import RemoteRunnable
rag_app = RemoteRunnable('http://0.0.0.0:8001/rag_chroma_private/')

this returns a generator -

rag_app.stream("How does agent memory work?")

and this returns the answer as a single item -

for item in rag_app.stream("How does agent memory work?"):
    print(item)

rlancemartin avatar Oct 27 '23 21:10 rlancemartin

Same problem here

gabegaz avatar Nov 15 '23 20:11 gabegaz

Same here. I got it to stream to stdout by adding a StreamingStdOutCallbackHandler, so the problem must be somewhere down the line.

funbotan avatar Dec 24 '23 11:12 funbotan