Eliran Cohen comments

Repositories
Issues
Comments

Results 3 comments of


                                            Eliran Cohen

Issue with concurrent requests on AWS Fargate

Hi @daixba, I forgot to mention that I'm not streaming the response With streaming, it works better, but it is still not perfect (I monitor the health-check endpoint, and it...

Issue with concurrent requests on AWS Fargate

@daixba when I run boto3 with asyncio it's working as expected https://github.com/aws-samples/bedrock-access-gateway/pull/23

run non-stream requests as async calls to avoid blocking requests

> For urgent usage, please do a customization to the code and deploy and host your own ECR images. Thanks, we are using custom deployment for EKS but I believe...