Connection for use sagemaker Endpoint

[theja0473]

Validations

• [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that requests the same enhancement

Problem

Currently Ollama or any self hosted services only allowed my suggestion lets connect to end point of Sagemaker and serve Copilot use case

Solution

No response

[sestinj]

Hi @theja0473 can you share more about what API format you expect for the Sagemaker endpoint?

[mikhail-khodorovskiy]

We are also interested in hosting the model in Sagemaker. The format of the API is:

https://runtime.sagemaker.us-west-2.amazonaws.com/endpoints/my-model-id/invocations

For boto, the documentation of how to invoke the endpoint: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime/client/invoke_endpoint.html

The credentials can be provided the same way as you have it implemented for bedrock: https://docs.continue.dev/reference/Model%20Providers/bedrock

[tobiasbartsch]

I am also very interested in hosting a model in Sagemaker. I think the main thing to add (as @mikhail-khodorovskiy pointed out) is credentials handling identical to how Bedrock is done.

[mikhail-khodorovskiy]

We actually were able to import the model we want into Bedrock where the model id but it still does not work: https://github.com/continuedev/continue/issues/1749