gateway icon indicating copy to clipboard operation
gateway copied to clipboard

feat: multimodal embeddings for bedrock titan and cohere

Open narengogi opened this issue 5 months ago • 1 comments
trafficstars

This PR adds support for multimodal embeddings in compliance with the portkey signature:

example requests:

  1. Titan text embedding:
curl --location 'http://localhost:8787/v1/embeddings' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-aws-access-key-id: ID' \
--header 'x-portkey-aws-secret-access-key: KEY' \
--header 'x-portkey-aws-region: us-east-1' \
--data '{
    "model": "amazon.titan-embed-text-v2:0",
    "input": "Hello this is a test",
    "normalize": false
}'
  1. Titan image embedding:
curl --location 'http://localhost:8787/v1/embeddings' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-aws-access-key-id: ID' \
--header 'x-portkey-aws-secret-access-key: KEY' \
--header 'x-portkey-aws-region: us-east-1' \
--data '{
    "model": "amazon.titan-embed-image-v1",
    "dimensions": 256,
    "input": [
        {
            "text": "this is the caption of the image",
            "image": {
                "base64": "UklGRkac....."
            }
        }
    ]
}'
  1. Cohere text embedding:
curl --location 'http://localhost:8787/v1/embeddings' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-aws-access-key-id: ID' \
--header 'x-portkey-aws-secret-access-key: KEY' \
--header 'x-portkey-aws-region: us-east-1' \
--data '{
    "model": "cohere.embed-english-v3",
    "input": ["Hello this is a test", "skibidi"],
    "input_type": "classification"
}'
  1. Cohere Multimodal embedding:
curl --location 'http://localhost:8787/v1/embeddings' \
--header 'x-portkey-provider: bedrock' \
--header 'Content-Type: application/json' \
--header 'x-portkey-aws-access-key-id: ID' \
--header 'x-portkey-aws-secret-access-key: KEY' \
--header 'x-portkey-aws-region: us-east-1' \
--data '{
    "model": "cohere.embed-english-v3",
    "input_type": "image",
    "dimensions": 256,
    "input": [
        {
            
            "image": {
                "base64": "UklGRkac....."
            }
        }
    ]
}'

narengogi avatar Jun 10 '25 12:06 narengogi