boto3 icon indicating copy to clipboard operation
boto3 copied to clipboard
verified publisher icon boto

Bedrock agent creation and execution not happening using inferenceConfiguration

Open pradeepdev-1995 opened this issue 1 month ago • 6 comments

Describe the bug

I want to create a bedrock agent with the foundation model anthropic.claude-3-7-sonnet and with model tuning parameters such as temperature, topP, stopSequences, etc.

tried the given code

import boto3
import uuid
aws_access_key = "KI6"
aws_secret_key = "Qc3"
aws_region     = "us-east-1"
def create_bedrock_agent_with_temperature(agent_name, agent_instruction, agent_role_arn, foundation_model, temperature=0.5):
    client = boto3.client('bedrock-agent',
                            region_name=aws_region,
                            aws_access_key_id=aws_access_key,
                            aws_secret_access_key=aws_secret_key)
    try:

        default_prompt_template = "{\"anthropic_version\":\"bedrock-2023-05-31\",\"system\":\"You are a helpful and professional assistant. Provide thorough, accurate answers in a formal tone.\",\"messages\":[{\"role\":\"user\",\"content\":\"$question$\"}]}"



        response = client.create_agent(
            agentName=agent_name,
            instruction=agent_instruction,
            foundationModel=foundation_model,
            agentResourceRoleArn=agent_role_arn,
            idleSessionTTLInSeconds=1800,
            promptOverrideConfiguration={
            'promptConfigurations': [
                {
                    'promptType': 'ORCHESTRATION', # Or ORCHESTRATION etc.
                    'promptCreationMode': 'OVERRIDDEN',  # OVERRIDDEN or DEFAULT
                    'promptState': 'ENABLED',
                    'basePromptTemplate': default_prompt_template,
                    'inferenceConfiguration': {
                        'temperature': 0.7, # This is where you set the temperature
                        'topP': 0.9,
                        'stopSequences': ['</response>'],
                        'maximumLength': 2048
                    }
                }
                ]
            }
        )
        print(f"Agent '{agent_name}' created successfully!")
        return response['agent']['agentArn']
    except Exception as e:
        print(f"Error creating agent: {e}")
        return None


role_arn = "arn:aws:iam::68296:role/DEFAULT_AgentExecutionRole"
model_id = "arn:aws:bedrock:us-east-1:68096:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0" 
# Create an agent with a higher temperature for more creative responses
creative_agent_arn = create_bedrock_agent_with_temperature(
    agent_name=f"CreativeAgent-{uuid.uuid4().hex[:8]}",
    agent_instruction="You are a creative writing assistant. Generate imaginative and diverse story ideas.",
    agent_role_arn=role_arn,
    foundation_model=model_id,
    temperature=0.8
)

print(f"Creative Agent ARN: {creative_agent_arn}")

But the agent creation is working, but the agent invocation is not happening, and it says

The overridden prompt that you provided is incorrectly formatted. Check the format for errors, such as invalid JSON, and retry your request.

So, as per my understanding

  • If we need to tune the model inside an agent, then we need to configure the inferenceConfiguration
  • To provide inferenceConfiguration the promptCreationMode value should be OVERRIDDEN
  • If we provide promptCreationMode value as OVERRIDDEN, we need to pass the basePromptTemplate

so what should be the basePromptTemplate value? How to structure this? Does this template differ based on the foundation model we used? Please provide any existing documentation for this?

Regression Issue

  • [ ] Select this option if this issue appears to be a regression.

Expected Behavior

Agent creation and invocation should happen smoothly with model tuning parameters

Current Behavior

Agent invocation failing.

Reproduction Steps

Use the code base above

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3==1.40.45, botocore==1.40.45

Environment details (OS name and version, etc.)

Ubuntu

pradeepdev-1995 avatar Nov 26 '25 06:11 pradeepdev-1995

Hello @pradeepdev-1995, thanks for reaching out. I have tried to replicate the issue using your code and did not get the same issue. For further investigation, could you please provide full debug logs by adding boto3.set_stream_logger('') after imports? Please redact any sensitive and security information. Thank you.

adev-code avatar Nov 26 '25 19:11 adev-code

@adev-code 2 points

  • The above code runs perfectly and will create the agent; there is no error. But once you try to invoke the created agent(using any hi/hello message), it fails and shows the error - The overridden prompt that you provided is incorrectly formatted. Check the format for errors, such as invalid JSON, and retry your request.

  • Here are the detailed logs after adding boto3.set_stream_logger('')

2025-11-27 04:47:40,140 botocore.hooks [DEBUG] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2025-11-27 04:47:40,141 botocore.hooks [DEBUG] Changing event name from before-call.apigateway to before-call.api-gateway
2025-11-27 04:47:40,142 botocore.hooks [DEBUG] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2025-11-27 04:47:40,142 botocore.hooks [DEBUG] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2025-11-27 04:47:40,142 botocore.hooks [DEBUG] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2025-11-27 04:47:40,143 botocore.hooks [DEBUG] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2025-11-27 04:47:40,143 botocore.hooks [DEBUG] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2025-11-27 04:47:40,145 botocore.hooks [DEBUG] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2025-11-27 04:47:40,145 botocore.hooks [DEBUG] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2025-11-27 04:47:40,145 botocore.hooks [DEBUG] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2025-11-27 04:47:40,145 botocore.hooks [DEBUG] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2025-11-27 04:47:40,159 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/endpoints.json
2025-11-27 04:47:40,203 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/sdk-default-configuration.json
2025-11-27 04:47:40,203 botocore.hooks [DEBUG] Event choose-service-name: calling handler <function handle_service_name_alias at 0x7186c2a36700>
2025-11-27 04:47:43,914 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/bedrock-agent/2023-06-05/service-2.json.gz
2025-11-27 04:47:46,803 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/bedrock-agent/2023-06-05/endpoint-rule-set-1.json.gz
2025-11-27 04:47:46,815 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/partitions.json
2025-11-27 04:47:46,817 botocore.hooks [DEBUG] Event creating-client-class.bedrock-agent: calling handler <function add_generate_presigned_url at 0x7186c2c937e0>
2025-11-27 04:47:46,817 botocore.configprovider [DEBUG] Looking for endpoint for bedrock-agent via: environment_service
2025-11-27 04:47:46,817 botocore.configprovider [DEBUG] Looking for endpoint for bedrock-agent via: environment_global
2025-11-27 04:47:46,817 botocore.configprovider [DEBUG] Looking for endpoint for bedrock-agent via: config_service
2025-11-27 04:47:46,817 botocore.configprovider [DEBUG] Looking for endpoint for bedrock-agent via: config_global
2025-11-27 04:47:46,817 botocore.configprovider [DEBUG] No configured endpoint found.
2025-11-27 04:47:46,817 botocore.regions [DEBUG] Creating a regex based endpoint for bedrock-agent, us-east-1
2025-11-27 04:47:46,818 botocore.endpoint [DEBUG] Setting bedrock-agent timeout as (60, 60)
2025-11-27 04:47:46,824 botocore.loaders [DEBUG] Loading JSON file: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/botocore/data/_retry.json
2025-11-27 04:47:46,824 botocore.client [DEBUG] Registering retry handlers for service: bedrock-agent
2025-11-27 04:47:46,825 botocore.hooks [DEBUG] Event before-parameter-build.bedrock-agent.CreateAgent: calling handler <function generate_idempotent_uuid at 0x7186c2a37f60>
2025-11-27 04:47:46,825 botocore.handlers [DEBUG] injecting idempotency token (5e77845c-6c50-4686-865a-61e71d9b4d78) into param 'clientToken'.
2025-11-27 04:47:46,825 botocore.hooks [DEBUG] Event before-parameter-build.bedrock-agent.CreateAgent: calling handler <function _handle_request_validation_mode_member at 0x7186c2a62b60>
2025-11-27 04:47:46,825 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'us-east-1', 'UseDualStack': False, 'UseFIPS': False}
2025-11-27 04:47:46,826 botocore.regions [DEBUG] Endpoint provider result: https://bedrock-agent.us-east-1.amazonaws.com  
2025-11-27 04:47:46,826 botocore.hooks [DEBUG] Event before-call.bedrock-agent.CreateAgent: calling handler <function add_recursion_detection_header at 0x7186c2a367a0>
2025-11-27 04:47:46,826 botocore.hooks [DEBUG] Event before-call.bedrock-agent.CreateAgent: calling handler <function add_query_compatibility_header at 0x7186c2a62ac0>
2025-11-27 04:47:46,826 botocore.hooks [DEBUG] Event before-call.bedrock-agent.CreateAgent: calling handler <function inject_api_version_header_if_needed at 0x7186c2a61a80>
2025-11-27 04:47:46,826 botocore.endpoint [DEBUG] Making request for OperationModel(name=CreateAgent) with params: {'url_path': '/agents/', 'query_string': {}, 'method': 'PUT', 'headers': {'Content-Type': 'application/json', 'User-Agent': 'Boto3/1.38.27 md/Botocore#1.38.27 ua/2.1 os/linux#6.6.87.2-microsoft-standard-WSL2 md/arch#x86_64 lang/python#3.11.13 md/pyimpl#CPython m/Z,b cfg/retry-mode#legacy Botocore/1.38.27'}, 'body': b'{"agentName": "CreativeAgent-c8d66873", "instruction": "You are a creative writing assistant. Generate imaginative and diverse story ideas.", "foundationModel": "arn:aws:bedrock:us-east-1:account_id:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0", "agentResourceRoleArn": "arn:aws:iam::account_id:role/DEFAULT_AgentExecutionRole", "idleSessionTTLInSeconds": 1800, "promptOverrideConfiguration": {"promptConfigurations": [{"promptType": "ORCHESTRATION", "promptCreationMode": "OVERRIDDEN", "promptState": "ENABLED", "basePromptTemplate": "{\\"anthropic_version\\":\\"bedrock-2023-05-31\\",\\"system\\":\\"You are a helpful and professional assistant. Provide thorough, accurate answers in a formal tone.\\",\\"messages\\":[{\\"role\\":\\"user\\",\\"content\\":\\"$question$\\"}]}", "inferenceConfiguration": {"temperature": 0.7, "topP": 0.9, "stopSequences": ["</response>"], "maximumLength": 2048}}]}, "clientToken": "5e77845c-6c50-4686-865a-61e71d9b4d78"}', 'url': 'https://bedrock-agent.us-east-1.amazonaws.com/agents/', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x7186c2180350>, 'has_streaming_input': False, 'auth_type': None, 'unsigned_payload': None}}
2025-11-27 04:47:46,827 botocore.hooks [DEBUG] Event request-created.bedrock-agent.CreateAgent: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7186c2180310>>
2025-11-27 04:47:46,827 botocore.hooks [DEBUG] Event choose-signer.bedrock-agent.CreateAgent: calling handler <function set_operation_specific_signer at 0x7186c2a37d80>
2025-11-27 04:47:46,827 botocore.auth [DEBUG] Calculating signature using v4 auth.
2025-11-27 04:47:46,827 botocore.auth [DEBUG] CanonicalRequest:
PUT
/agents/

content-type:application/json
host:bedrock-agent.us-east-1.amazonaws.com
x-amz-date:20251127T044746Z

content-type;host;x-amz-date
9dcfebc1f05a5d5a79d68e881d47dd252ad7f8cdda2ca5883f4f3b4425d34e10
2025-11-27 04:47:46,827 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20251127T044746Z
20251127/us-east-1/bedrock/aws4_request
a1c224c2f846d4fc6aff00b2545fc698edc9b85ee0dcf0fa923bd7b6691d33de
2025-11-27 04:47:46,828 botocore.auth [DEBUG] Signature:
5bd19f3def7d3b17cbc6d2d1100ff60178da03e5883753a5e4c809f391036f5c
2025-11-27 04:47:46,828 botocore.hooks [DEBUG] Event request-created.bedrock-agent.CreateAgent: calling handler <bound method UserAgentString.rebuild_and_replace_user_agent_handler of <botocore.useragent.UserAgentString object at 0x7186c2e82e50>>
2025-11-27 04:47:46,828 botocore.hooks [DEBUG] Event request-created.bedrock-agent.CreateAgent: calling handler <function add_retry_headers at 0x7186c2a62340>
2025-11-27 04:47:46,828 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://bedrock-agent.us-east-1.amazonaws.com/agents/, headers={'Content-Type': b'application/json', 'User-Agent': b'Boto3/1.38.27 md/Botocore#1.38.27 ua/2.1 os/linux#6.6.87.2-microsoft-standard-WSL2 md/arch#x86_64 lang/python#3.11.13 md/pyimpl#CPython m/Z,b cfg/retry-mode#legacy Botocore/1.38.27', 'X-Amz-Date': b'20251127T044746Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=api_key/20251127/us-east-1/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=5bd19f3def7d3b17cbc6d2d1100ff60178da03e5883753a5e4c809f391036f5c', 'amz-sdk-invocation-id': b'5e1c1b70-79ed-44f4-b065-80f3d2a5b145', 'amz-sdk-request': b'attempt=1', 'Content-Length': '958'}>
2025-11-27 04:47:46,829 botocore.httpsession [DEBUG] Certificate path: /mnt/c/Users/PRADEEPT/Documents/GitHub/repo_name/.venv/lib/python3.11/site-packages/certifi/cacert.pem
2025-11-27 04:47:46,830 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): bedrock-agent.us-east-1.amazonaws.com:443
2025-11-27 04:47:48,341 urllib3.connectionpool [DEBUG] https://bedrock-agent.us-east-1.amazonaws.com:443 "PUT /agents/ HTTP/1.1" 202 1206
2025-11-27 04:47:48,341 botocore.parsers [DEBUG] Response headers: {'Date': 'Thu, 27 Nov 2025 04:47:51 GMT', 'Content-Type': 'application/json', 'Content-Length': '1206', 'Connection': 'keep-alive', 'x-amzn-RequestId': '7b53f718-af43-4def-9987-21e3afc01635', 'x-amz-apigw-id': 'Ur6usG5CIAMEtcw=', 'X-Amzn-Trace-Id': 'Root=1-6927d7f7-2a6d614e21ccabed1c640006'}       
2025-11-27 04:47:48,341 botocore.parsers [DEBUG] Response body:
b'{"agent":{"agentArn":"arn:aws:bedrock:us-east-1:account_id:agent/KOHJQOBMDE","agentCollaboration":"DISABLED","agentId":"KOHJQOBMDE","agentName":"CreativeAgent-c8d66873","agentResourceRoleArn":"arn:aws:iam::account_id:role/DEFAULT_AgentExecutionRole","agentStatus":"CREATING","createdAt":"2025-11-27T04:47:51.718637608Z","foundationModel":"arn:aws:bedrock:us-east-1:account_id:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0","idleSessionTTLInSeconds":1800,"instruction":"You are a creative writing assistant. Generate imaginative and diverse story ideas.","orchestrationType":"DEFAULT","promptOverrideConfiguration":{"promptConfigurations":[{"basePromptTemplate":"{\\"anthropic_version\\":\\"bedrock-2023-05-31\\",\\"system\\":\\"You are a helpful and professional assistant. Provide thorough, accurate answers in a formal tone.\\",\\"messages\\":[{\\"role\\":\\"user\\",\\"content\\":\\"$question$\\"}]}","inferenceConfiguration":{"maximumLength":2048,"stopSequences":["</response>"],"temperature":0.699999988079071,"topP":0.8999999761581421},"parserMode":"DEFAULT","promptCreationMode":"OVERRIDDEN","promptState":"ENABLED","promptType":"ORCHESTRATION"}]},"updatedAt":"2025-11-27T04:47:51.718637608Z"}}'
2025-11-27 04:47:48,342 botocore.hooks [DEBUG] Event needs-retry.bedrock-agent.CreateAgent: calling handler <botocore.retryhandler.RetryHandler object at 0x7186c2902750>
2025-11-27 04:47:48,343 botocore.retryhandler [DEBUG] No retry needed.
Agent 'CreativeAgent-c8d66873' created successfully!
Creative Agent ARN: arn:aws:bedrock:us-east-1:account_id:agent/KOHJQOBMDE

pradeepdev-1995 avatar Nov 27 '25 04:11 pradeepdev-1995

@adev-code any updates?

pradeepdev-1995 avatar Dec 01 '25 12:12 pradeepdev-1995

Thanks for the reply. I have tried to replicate the issue and it seems that the service is giving the error:

botocore.parsers [DEBUG] Response body:
b'{"message":"The overridden prompt that you provided is incorrectly formatted. Check the format for errors, such as invalid JSON, and retry your request."}'

I have reached out to the Bedrock service team in this regard, what has caused the issue, the fix and answers to your questions. I will update as soon as there are any updates from the team.

Internal Ref: P347133937

adev-code avatar Dec 02 '25 19:12 adev-code

@adev-code Thanks

pradeepdev-1995 avatar Dec 03 '25 02:12 pradeepdev-1995

@adev-code any further updates?

pradeepdev-1995 avatar Dec 08 '25 08:12 pradeepdev-1995