boto
Sorry, I'm unable to assist you with this request
Describe the bug
When using the boto3 retrieve_and_generate method with knowledge bases, we're experiencing intermittent failures where the API returns a generic error message: "Sorry, I'm unable to assist you with this request" with a 200 OK status code, despite the underlying LLM (Claude 3 Haiku/Sonnet) generating a valid response.
Root Cause Analysis
- Through troubleshooting, we have identified that the issue occurs when:
- The prompt template includes the
$output_format_instructions$placeholder (required for citations) - The LLM's response doesn't perfectly conform to the expected XML structure
- Bedrock's internal XML parser fails to parse the response
- Instead of returning the actual LLM output, the service returns the generic error message
Expected Behavior
Bedrock should respond with whatever LLM gives back.
Current Behavior
Bedrock returns Sorry, I'm unable to assist you with this request
Reproduction Steps
This generally happens whenever i ask a query for which there is no relevant information in my documents.
Example -
User - Your competitor XYZ charges $262 a year without a renewal fee.
Bedrock - Sorry, I'm unable to assist you with this request.
After removing the $output_format_instructions$ from the system prompt
Bedrock - I apologize, but I cannot discuss or compare our pricing with competitors. Each XYZ service has its own unique pricing structure. I can help you understand XYZ's services and charges. Would you like to know more about our current service rates or additional fees? I'd be happy to provide information about XYZ's pricing and service options.
SDK version used
1.40.34
Environment details (OS name and version, etc.)
macOS Sequoia 15.6.1, Lambda Python 3.13
A Follow up issue on this - If you remove the parameter $output_format_instructions$ then ideally you should not get any citations as per the documentation but below is the response we get.
...
...
"citations": [
{
"generatedResponsePart": {
"textResponsePart": {
"span": {
"end": 50,
"start": 0
},
"text": "Sorry, I am unable to assist you with this request."
}
},
"retrievedReferences": []
}
],
...
...
Expected - Empty array/list
I think it triggers the code to get the citations internally even though it shouldn't. It does not get the citations from LLM and falls back to its default message.
Hello @shubsaini09, thanks for reaching out. Using the boto3.set_stream_logger('') to see the logs, could you please provide the full logs especially the full Response body and Response headers. Please redact any sensitive information. Also, please provide a minimal reproducible code for the behavior.
@adev-code I want to let you know that i enabled model invocation logging in bedrock and i can clearly see the LLM response coming back but restricted by boto3. This issue seems to be affecting multiple customers. Please refer to this helpful blog amazon-bedrocks-sorry-im-unable-to-assist-you-with-this-request-solved-a-journey-into-troubleshooting-and-resolution
Below are the logs generated after setting boto3.set_stream_logger('').
Enter your query: Your competitor XYZ charges $262 a year without a renewal fee
2025-09-22 14:49:43,506 __main__ [INFO] Event: {"query": "Your competitor XYZ charges $262 a year without a renewal fee"}
2025-09-22 14:49:43,507 botocore.hooks [DEBUG] Event before-parameter-build.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function generate_idempotent_uuid at 0x102ed4c20>
2025-09-22 14:49:43,507 botocore.hooks [DEBUG] Event before-parameter-build.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function _handle_request_validation_mode_member at 0x102ed7740>
2025-09-22 14:49:43,507 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'us-east-1', 'UseDualStack': False, 'UseFIPS': False}
2025-09-22 14:49:43,508 botocore.regions [DEBUG] Endpoint provider result: https://bedrock-agent-runtime.us-east-1.amazonaws.com
2025-09-22 14:49:43,509 botocore.hooks [DEBUG] Event before-call.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function add_recursion_detection_header at 0x102ed4720>
2025-09-22 14:49:43,509 botocore.hooks [DEBUG] Event before-call.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function inject_api_version_header_if_needed at 0x102ed6700>
2025-09-22 14:49:43,509 botocore.endpoint [DEBUG] Making request for OperationModel(name=RetrieveAndGenerate) with params: {'url_path': '/retrieveAndGenerate', 'query_string': {}, 'method': 'POST', 'headers': {'Content-Type': 'application/json', 'User-Agent': 'Boto3/1.40.32 md/Botocore#1.40.32 ua/2.1 os/macos#24.6.0 md/arch#arm64 lang/python#3.13.5 md/pyimpl#CPython m/g,Z,b,E cfg/retry-mode#standard Botocore/1.40.32'}, 'body': b'{"input": {"text": "Your competitor XYZ charges $262 a year without a renewal fee"}, "retrieveAndGenerateConfiguration": {"type": "KNOWLEDGE_BASE", "knowledgeBaseConfiguration": {"knowledgeBaseId": "", "modelArn": "arn:aws:bedrock:us-east-1::inference-profile/us.anthropic.claude-3-5-haiku--v1:0", "generationConfiguration": {"inferenceConfig": {"textInferenceConfig": {"temperature": 0.0, "topP": 0.7, "maxTokens": 512}}, "performanceConfig": {"latency": "optimized"}, "promptTemplate": {"textPromptTemplate": "my system prompt. OFFICIAL DOCUMENTATION: $search_results$\\n\\n$output_format_instructions$\\n"}, "guardrailConfiguration": {"guardrailId": "", "guardrailVersion": "1"}}}}}', 'url': 'https://bedrock-agent-runtime.us-east-1.amazonaws.com/retrieveAndGenerate', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x102f9da90>, 'has_streaming_input': False, 'auth_type': None, 'unsigned_payload': None, 'auth_options': ['aws.auth#sigv4']}}
2025-09-22 14:49:43,509 botocore.hooks [DEBUG] Event request-created.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x102f8af90>>
2025-09-22 14:49:43,509 botocore.hooks [DEBUG] Event choose-signer.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function set_operation_specific_signer at 0x102ed49a0>
2025-09-22 14:49:43,509 botocore.hooks [DEBUG] Event choose-signer.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function _set_auth_scheme_preference_signer at 0x102ed7880>
2025-09-22 14:49:43,510 botocore.auth [DEBUG] Calculating signature using v4 auth.
2025-09-22 14:49:43,510 botocore.auth [DEBUG] CanonicalRequest:
POST
/retrieveAndGenerate
content-type:application/json
host:bedrock-agent-runtime.us-east-1.amazonaws.com
x-amz-date:20250922T194943Z
2025-09-22 14:49:43,510 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20250922T194943Z
20250922/us-east-1/bedrock/aws4_request
2025-09-22 14:49:43,510 botocore.auth [DEBUG] Signature:
2025-09-22 14:49:43,510 botocore.hooks [DEBUG] Event request-created.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <bound method UserAgentString.rebuild_and_replace_user_agent_handler of <botocore.useragent.UserAgentString object at 0x102f9e850>>
2025-09-22 14:49:43,510 botocore.hooks [DEBUG] Event request-created.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <function add_retry_headers at 0x102ed6fc0>
2025-09-22 14:49:43,510 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://bedrock-agent-runtime.us-east-1.amazonaws.com/retrieveAndGenerate, headers={'Content-Type': b'application/json', 'User-Agent': b'Boto3/1.40.32 md/Botocore#1.40.32 ua/2.1 os/macos#24.6.0 md/arch#arm64 lang/python#3.13.5 md/pyimpl#CPython m/g,Z,b,E cfg/retry-mode#standard Botocore/1.40.32', 'X-Amz-Date': b'20250922T194943Z', 'X-Amz-Security-Token': b'', 'Authorization': b'AWS4-HMAC-SHA256 Credential=/20250922/us-east-1/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=', 'amz-sdk-invocation-id': b'', 'amz-sdk-request': b'attempt=1', 'Content-Length': '1412'}>
2025-09-22 14:49:43,522 botocore.httpsession [DEBUG] Certificate path: /opt/anaconda3/lib/python3.13/site-packages/certifi/cacert.pem
2025-09-22 14:49:43,523 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): bedrock-agent-runtime.us-east-1.amazonaws.com:443
2025-09-22 14:49:46,373 urllib3.connectionpool [DEBUG] https://bedrock-agent-runtime.us-east-1.amazonaws.com:443 "POST /retrieveAndGenerate HTTP/1.1" 200 326
2025-09-22 14:49:46,374 botocore.parsers [DEBUG] Response headers: {'Date': 'Mon, 22 Sep 2025 19:49:46 GMT', 'Content-Type': 'application/json', 'Content-Length': '326', 'Connection': 'keep-alive', 'x-amzn-RequestId': ''}
2025-09-22 14:49:46,374 botocore.parsers [DEBUG] Response body:
b'{"citations":[{"generatedResponsePart":{"textResponsePart":{"span":{"end":50,"start":0},"text":"Sorry, I am unable to assist you with this request."}},"retrievedReferences":[]}],"guardrailAction":"NONE","output":{"text":"Sorry, I am unable to assist you with this request."},"sessionId":""}'
2025-09-22 14:49:46,374 botocore.hooks [DEBUG] Event needs-retry.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1037bd940>>
2025-09-22 14:49:46,374 botocore.retries.standard [DEBUG] Not retrying request.
2025-09-22 14:49:46,375 botocore.hooks [DEBUG] Event after-call.bedrock-agent-runtime.RetrieveAndGenerate: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x1037bc830>>
2025-09-22 14:49:46,375 src.bedrock [INFO] Bedrock response: status=200
My code is very simple and basic. I am using retrieve_and_generate to get the answer back for my query. Below are the configurations that I used.
response = _bedrock_agent_runtime.retrieve_and_generate(
input={
'text': prompt
},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': KNOWLEDGE_BASE_ID,
'modelArn': MODEL_ARN,
"generationConfiguration": {
"inferenceConfig": {
"textInferenceConfig": {
"temperature": TEMPERATURE,
"topP": TOP_P,
"maxTokens": MAX_TOKENS,
}
},
"performanceConfig": {
"latency": "optimized"
},
'promptTemplate': {
'textPromptTemplate': SYSTEM_PROMPT_TEMPLATE
},
"guardrailConfiguration": {
"guardrailId": GUARDRAIL_ID,
"guardrailVersion": GUARDRAIL_VERSION
},
}
}
}
)
Thank you for the response and all information. Since the Bedrock service is giving the response and the behavior, I have reached out to the bedrock Service Team in this regards. An update will be given once the team has given an update.
Internal Ref: P305485185