Unable to Force Portkey to Use Bedrock Converse Endpoint

[dldereklee]

What Happened?

I cannot seem to figure out why Portkey sometimes decides to use the /invoke-with-response-stream instead of /converse-stream endpoint for Bedrock models . The /invoke-with-response-stream url does not work for our AWS account so I need to redirect all calls to /converse-stream

Looking at the Portkey docs, there doesn't seem to be a parameter that I can pass to specify which API it should go to.

What Should Have Happened?

There should be a parameter to specify which Bedrock API should be used for the model.

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response

[narengogi]

Hey @dldereklee yes, you can make the requests work through portkey for invoke, you can simply set this header x-portkey-custom-host: https://bedrock-runtime.{your region}.amazonaws.com/model/{your model url encode if necessary}/invoke and make a request to /v1 . This will allow you to use portkey for authentication (IRSA/secret and access key, assumed role etc) but you will be required to send body in the invoke format itself

Open Source

curl --location 'http://localhost:8787/v1' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-aws-access-key-id: {}' \
--header 'x-portkey-aws-secret-access-key: {}' \
--header 'x-portkey-aws-region: {REGION}' \
--header 'x-portkey-custom-host: https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-3-7-sonnet-20250219-v1%3A0/invoke' \
--data '{
    "messages": [
        {
            "role": "user",
            "content": "How are you doing sir?"
        }
    ],
    "max_tokens": 100,
    "anthropic_version": "bedrock-2023-05-31"
}'

SaaS

curl --location 'https://spi.portkey.ai/v1' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-api-key: {PORTKEY_API_KEY}' \
--header 'x-portkey-virtual-key: {VIRTUAL_KEY}' \
--header 'x-portkey-custom-host: https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-3-7-sonnet-20250219-v1%3A0/invoke' \
--data '{
    "messages": [
        {
            "role": "user",
            "content": "How are you doing sir?"
        }
    ],
    "max_tokens": 100,
    "anthropic_version": "bedrock-2023-05-31"
}'

We'd migrated to /converse a while ago for maintainability since aws maintains it upstream.

[dldereklee]

When I try using x-portkey-custom-host and provide the /converse endpoint, Portkey appends /model/us.anthropic.claude-sonnet-4-20250514-v1%3A0/invoke-with-response-stream to it.

So then in the Portkey logs, I see the url it is trying to use is

"url": "https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-sonnet-4-20250514-v1%3A0/converse/model/us.anthropic.claude-sonnet-4-20250514-v1%3A0/invoke-with-response-stream",

[narengogi]

When I try using x-portkey-custom-host and provide the /converse endpoint, Portkey appends /model/us.anthropic.claude-sonnet-4-20250514-v1%3A0/invoke-with-response-stream to it.

@dldereklee Are you calling /v1 or /v1/chat/completions? If you're using an openai compatible sdk, you need to be invoking client.post() instead of client.chat.completions() since you want to be using the gateway only for auth and not as a unified transformation route!

Please let us know if this doesnt help, we're also available on the discord server for quick turnaround https://discord.gg/8vT2hz3fdG