cogyair
cogyair
I suspect that generate_iterator function consumed by :self.runner.generate_iterator.async_stream in _llm.py assumes the stream brings one message at a time while it can send a chunk of them. maybe it is...
what did you do to resolve? (I moved out of openllm for now..) also saw this error here: https://github.com/langchain-ai/langchain/discussions/15744
I am also not a rust developer but you can see how to add it in this example: https://pyo3.rs/v0.20.2/parallelism