Add Amazon Bedrock Unified Chat Completions support

[Evgenii-Kazannik]

{{base-url}}/_inference/chat_completion/amazonbedrock_inference_id_chat

{ "service": "amazonbedrock", "service_settings": { "access_key": "{{AWS-access-key}}", "secret_key": "{{AWS-secret-key}}", "region": "us-east-1", "provider": "anthropic", "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0" } }

{{base-url}}/_inference/chat_completion/amazonbedrock_inference_id_chat/_stream

{ "messages": [ { "role": "user", "content": "What the weather is in Boston?" } ], "max_completion_tokens": 1000, "top_p": 0.1, "tools": [ { "type": "any", "function": { "name": "get_current_weather", "description": "Get current weather", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" }, "units": { "type": "string", "enum": [ "celsius", "fahrenheit" ], "description": "Units the temperature will be returned in." } } } } } ] }

response for the request with tool calling

message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ius\"}"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.330 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ls"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.328 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\": \"ce"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.328 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ts"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.327 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":", \"uni"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.327 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"USA\""},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.326 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"n, "},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.320 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"sto"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.319 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"tion\": \"Bo"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.317 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"loca"},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.316 message {"id":null,"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":""},"type":"TOOL_USE"}]},"index":0}],"model":null,"object":null} 17:21:13.314 message {"id":null,"choices":[{"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"tooluse_Z7IP83_eTt2y_TECni1ULw","function":{"name":"get_current_weather"},"type":"get_current_weather"}]},"index":0}], 17:21:13.298 message {"id":null,"choices":[{"delta":{"role":"assistant"},"index":0}],"model":null,"object":null} 17:21:13.296

the model returns the tool use id like: tooluse_Z7IP83_eTt2y_TECni1ULw which is to be used when returning tool results to the model

{{base-url}}/_inference/chat_completion/amazonbedrock_inference_id_chat/_stream

{ "messages": [ { "role": "assistant", "content": "What the weather is in Boston?", "tool_calls": [ { "id": "tooluse_ya1POnYQRu-_cWACiBfsHw", "type": "tool_call", "function": { "name": "get_current_weather", "arguments": "{\"location\":\"Boston, USA\"}" } } ] }, { "role": "user", "content": "0 degree", "tool_call_id": "tooluse_ya1POnYQRu-_cWACiBfsHw" } ] }

response for the request with tool results

message {"id":null,"choices":[{"delta":{"content":" there, please dress very warmly with"},"index":0}],"model":null,"object":null} 17:31:54.186 message {"id":null,"choices":[{"delta":{"content":" Boston or planning to go"},"index":0}],"model":null,"object":null} 17:31:54.018 message {"id":null,"choices":[{"delta":{"content":").\n\nIf you're in"},"index":0}],"model":null,"object":null} 17:31:53.836 message {"id":null,"choices":[{"delta":{"content":" cold (-17.8°C"},"index":0}],"model":null,"object":null} 17:31:53.704 message {"id":null,"choices":[{"delta":{"content":" Fahrenheit, that's extremely"},"index":0}],"model":null,"object":null} 17:31:53.565 message {"id":null,"choices":[{"delta":{"content":"°F). If it's in"},"index":0}],"model":null,"object":null} 17:31:53.439 message {"id":null,"choices":[{"delta":{"content":" the freezing point of water (32"},"index":0}],"model":null,"object":null} 17:31:53.399 message {"id":null,"choices":[{"delta":{"content":", it's exactly at"},"index":0}],"model":null,"object":null} 17:31:53.398 message {"id":null,"choices":[{"delta":{"content":" that's in Celsius"},"index":0}],"model":null,"object":null} 17:31:53.396 message {"id":null,"choices":[{"delta":{"content":"ing temperature! If"},"index":0}],"model":null,"object":null} 17:31:53.395 message {"id":null,"choices":[{"delta":{"content":" in Boston. That's freez"},"index":0}],"model":null,"object":null} 17:31:53.395 message {"id":null,"choices":[{"delta":{"content":" it's currently 0 degrees"},"index":0}],"model":null,"object":null} 17:31:53.267 message {"id":null,"choices":[{"delta":{"content":" you for letting me know that"},"index":0}],"model":null,"object":null} 17:31:53.267 message {"id":null,"choices":[{"delta":{"content":" my confusion earlier. Thank"},"index":0}],"model":null,"object":null} 17:31:53.266 message {"id":null,"choices":[{"delta":{"content":"I apologize for"},"index":0}],"model":null,"object":null} 17:31:53.266 message {"id":null,"choices":[{"delta":{"role":"assistant"},"index":0}],"model":null,"object":null} 17:31:53.266

[elasticsearchmachine]

Pinging @elastic/search-inference-team (Team:Search - Inference)

[dan-rubinstein]

@elasticmachine test this please

[jonathan-buttner]

@elasticmachine test this please

[jonathan-buttner]

@Evgenii-Kazannik can you give me and Dan write access to your repo?

I'd like to make some changes (like adding the change log). If you can't, then could you add this file:

docs/changelog/133697.yaml

pr: 133697
summary: Added Amazon Bedrock chat completion support to the Inference API
area: Machine Learning
type: enhancement
issues: []

[jonathan-buttner]

Closing in favor of this one: https://github.com/elastic/elasticsearch/pull/139411