bedrock-access-gateway icon indicating copy to clipboard operation
bedrock-access-gateway copied to clipboard

Cannot handle properly when bedrock response with empty string

Open Shellmode opened this issue 6 months ago • 2 comments

Describe the bug 当 bedrock 返回空字符串的时候,在 https://github.com/aws-samples/bedrock-access-gateway/blob/main/src/api/models/bedrock.py#L456 处会报错,trace 如下

image

Please complete the following information:

  • [x] Which API you used: [e.g. /chat/completions]
  • [] Which model you used: [e.g. anthropic.claude-3-sonnet-20240229-v1:0]

To Reproduce Steps to reproduce the behavior. If possible, please share an example request.

curl http://Bedroc-Proxy-abcdefg-123123.us-west-2.elb.amazonaws.com/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer bedrock" \
  -d '{
    "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "messages": [
      {"role": "user", "content": "Given a [text fragment], extract the person names from it. Note that: 1. List the person names directly, separated by commas, like this: Mary, Mike, Lucy. 2. If no characters are recognized in the [text fragment], return an empty string directly, without adding any explanations or comments. 3. Do not output any additional text or explanations. Text fragment: \"Darkness.\"He suddenly woke up, and that was his first thought.\" Extracting result:"}
    ]
  }'

Expected behavior Response with empty string

Screenshots image

Additional context Add any other context about the problem here.

Shellmode avatar Aug 09 '24 09:08 Shellmode