feat: Enable Imported models

[sean-smith]

This allows users to call models they've imported (if enabled by an environment variable):

For example:

from openai import OpenAI

client = OpenAI()
completion = client.chat.completions.create(
    model="arn:aws:bedrock:us-west-2:<account-id>:imported-model/<model-id>",
    # model="meta.llama3-3-70b-instruct-v1:0",
    messages=[
        {
            "role": "user",
            "content": "Hello! please tell me a joke"
        }
    ],
)

These models also show up in the model list:

client.models.list()

This resolves https://github.com/aws-samples/bedrock-access-gateway/issues/99

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[asadafa123]

Can this also support models deployed with sagemaker endpoint? Not sure because in aws console the arn are like: arn:aws:sagemaker:us-east-1:ACCOUNT_ID:endpoint/endpoint-quick-start-pu6a8, arn:aws:sagemaker:us-east-1:aws:hub-content/SageMakerPublicHub/Model/deepseek-llm-r1-distill-qwen-7b/1.0.0

[daixba]

Thanks for the contribution.

I am afraid I can't merge the code for now. The repo is using converse API to access Bedrock Models. But right now, there are very limited support on custom imported models for the converse API. So it's confusing to have such feature included at this stage. We may also want to deal with the cold start of imported models in the future.

[sean-smith]

Can this also support models deployed with sagemaker endpoint? Not sure because in aws console the arn are like: arn:aws:sagemaker:us-east-1:ACCOUNT_ID:endpoint/endpoint-quick-start-pu6a8, arn:aws:sagemaker:us-east-1:aws:hub-content/SageMakerPublicHub/Model/deepseek-llm-r1-distill-qwen-7b/1.0.0

No, this is limited to just bedrock model imports (which is different from bedrock fine tuned models).

Thanks for the contribution.

I am afraid I can't merge the code for now. The repo is using converse API to access Bedrock Models. But right now, there are very limited support on custom imported models for the converse API. So it's confusing to have such feature included at this stage. We may also want to deal with the cold start of imported models in the future.

@daixba It would be useful to see the roadmap for custom model import. Right now the integration is very much lacking and it seems like the bedrock is not investing into custom model import.