Detailed alternative to `Estimator.deploy(update_endpoint=True)`

[oliver-leung]

What did you find confusing? Please describe. I am trying to retrain a SKLearn estimator in SageMaker, then redeploy it to an existing inference endpoint. However, the update_endpoint=True option in the Estimator.deploy() function has been deprecated, and it isn't immediately clear how one would accomplish it from the Predictor.update_endpoint() function

Describe how documentation can be improved This entry seems to be the most recent announcement for the deprecation, but the documentation page that it points to doesn't give a good description of how one would update the endpoint given the

Additional context I managed to update the existing endpoint with a new model from the solution described in #58, but I was hoping that there would be an easier way to reference the new model.

[jeniyat]

Thanks for the suggestions.

[caitriggs]

Similar issue but with ModelPackage instead of the Estimator class. I'm reinstantiating a model using sagemaker.ModelPackage() so that I can deploy a registered model in Model Registry.

Deploying the ModelPackage model as an endpoint works, but then when I want to replace the underlying deployed model ModelPackage.deploy(update_endpoint=True) fails and there's no obvious other option to update endpoint.

Trying to reinstantiate a Predictor object and passing a model_name in also does not seem to be the answer to update the underlying model on the endpoint, since model_name doesn't take a ModelPackageARN. How can we update the underlying model version on the existing endpoint?

Documentation on options for updating an existing endpoint with a new model are unclear.

E.g. Deploy the model version in Model Registry

from sagemaker import ModelPackage

model_package_arn = 'arn:aws:sagemaker:us-east-2:12345678901:model-package/modeltest/1'
model = ModelPackage(role=role, 
                     model_package_arn=model_package_arn, 
                     sagemaker_session=sagemaker_session)
model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge', endpoint_name='my-endpoint')

But cannot update the endpoint by passing a new model package ARN bc this fails wanting a simple regex-approved string

from sagemaker.predictor import Predictor
predictor = Predictor(endpoint_name='my-endpoint', sagemaker_session=sagemaker_session_object)
predictor.update_endpoint(instance_type="ml.t2.large", initial_instance_count=1, model_name=some_new_model_package_arn)

[l3ku]

+1 here. Our workflow is to train new versions of models into a model package group, and be able to replace the existing endpoint which uses the previous model with the new model, while retaining the same endpoint name.

Currently I am resorting to boto3 to update the existing endpoint, but it involves quite many steps:

client = boto3.client("sagemaker")
client.create_model(**model_params)
client.create_endpoint_config(**endpoint_config_params)

try:
    client.create_endpoint(**endpoint_params)
except botocore.exceptions.ClientError as err:
    err_message = err.response["Error"]["Message"]
    if err_message.startswith("Cannot create already existing endpoint"):
        logger.info("Endpoint already exists, updating configuration")
        client.update_endpoint(**endpoint_params)

Not speaking of the overhead to always generate unique IDs (based on model package version or timestamp) for the new model and endpoint configurations.

[clausagerskov]

@jeniyat any update on this?

[mennohofste]

I ran into the same issue. I wanted to update an existing endpoint with a SageMaker (PyTorch) model obtained from the Estimator class.

I came up with the following solution:

instance_type = 'ml.m5.large'
model_name = 'a-suitable-model-name'
model = PyTorchModel(...)

# Creates a model in the SageMaker model section
container_defs = model.prepare_container_def(instance_type)
sagemaker_session.create_model(name=model_name, role=role, container_defs=container_defs)

# Update the endpoint using the Predictor
predictor = Predictor(endpoint_name='my-endpoint', sagemaker_session=sagemaker_session)
predictor.update_endpoint(initial_instance_count=1, instance_type=instance_type, model_name=model_name)

The prepare_container_def() function changes slightly for different model types, so you might have to give more or less parameters depending.

It works flawless for our purposes so far. I hope this helps someone.

[rodrigofp-possiblefinance]

Any update on this?

It's not a pleasant approach to always create a new model and endpoint config and then deleting previous model and configs. When creating or updating a model the model version should be updated as well, without the need to use model registry explicitly. It should be under the hood.

[taepd]

It seems like we could easily solve the update_endpoint problem by allowing endpoint_name and enndpoint_configuration_name to be entered as parameters at the same time during model.deploy. Is there any reason not to do this?

[gorj-tessella]

There should be a simple approach for updating the model/endpoint with a given name. Why was update_endpoint removed in the first place?