langchain4j icon indicating copy to clipboard operation
langchain4j copied to clipboard

fix ollama client response handle

Open HashJang opened this issue 10 months ago • 2 comments

Thanks for @wangrushuang wangrushuang, she find this problem and I provide a way to fix that.

HashJang avatar Apr 01 '24 03:04 HashJang

@HashZhang please provide some context: what is the problem and how do you fix it? Please fill in the PR template. Thank you.

langchain4j avatar Apr 03 '24 08:04 langchain4j

Sorry, the problem is that the original code uses a fixed length byte array buffer for the stream response. If one of the stream response is over 1024 then the last json may miss some subparts. Therefore we should use some separator to split the response and streaming deserialize it. Use BufferedReader would be a way to fix that.

HashJang avatar Apr 04 '24 13:04 HashJang

@HashZhang how did you test/reproduce the error? Thanks

langchain4j avatar Apr 08 '24 15:04 langchain4j