langchain-aws icon indicating copy to clipboard operation
langchain-aws copied to clipboard

Published 20 hours ago • verified publisher icon langchain-ai

Bug: Fix recursion error when using streaming with tools

Open munday-tech opened this issue 1 year ago • 8 comments

Added additional param to allow independent turning off for streaming to ensure recursion errors don't occur when calling generate from stream with streaming=True on BedrockChat

munday-tech avatar Jun 21 '24 13:06 munday-tech

Fixes #80

munday-tech avatar Jun 21 '24 13:06 munday-tech

@3coins @bigbernnn can you please have a look? I confirmed this PR is working

theagenticguy avatar Jun 24 '24 17:06 theagenticguy

@munday-tech Looks good! Please fix small linting error.

bigbernnn avatar Jun 25 '24 18:06 bigbernnn

@bigbernnn Should be fixed now.

munday-tech avatar Jun 26 '24 00:06 munday-tech

Is there any update on this? I am facing the same issue

deepakdalakoti avatar Jul 11 '24 11:07 deepakdalakoti

I did a quick test with this solution. While this fixes the recursion bug, the tokens are still not being streamed (for claude-3 models)

deepakdalakoti avatar Jul 12 '24 04:07 deepakdalakoti

@munday-tech could you review the changes requested and also confirm if the tokens get streamed with this fix ?

rsgrewal-aws avatar Jul 22 '24 15:07 rsgrewal-aws

Working on another PR, that allows streaming to work as expected. The change here uses interplay between _stream and _generate and does not return the chunks independently. See https://github.com/langchain-ai/langchain-aws/pull/119

3coins avatar Jul 23 '24 17:07 3coins

Fixed by #119

3coins avatar Oct 07 '24 04:10 3coins