sagemaker-python-sdk icon indicating copy to clipboard operation
sagemaker-python-sdk copied to clipboard

update_endpoint does not support serverless_inference_config

Open WillBrennan opened this issue 1 year ago • 1 comments

Currently you can deploy a serverless endpoint with,

predictor = model.deploy(serverless_inference_config=ServerlessInferenceConfig())

however serverless_inference_config is not supported when you call predictor.update_endpoint you have to use a standard endpoint;

predictor.update_endpoint(model_name=model.name, instance_type="ml.m4.xlarge", initial_instance_count=1)` 

. Ideally you would be able to call,

predictor.update_endpoint(model_name=model.name, serverless_inference_config=ServerlessInferenceConfig())

to keep a serverless deployment.

WillBrennan avatar Sep 21 '22 13:09 WillBrennan

Similarly, it'd be great if we can add serverless_inference_config kwargs in sagemaker.session.Session.create_endpoint_config

class Session(object):  
    def create_endpoint_config(
        self,
        name,
        model_name,
        initial_instance_count,
        instance_type,
        accelerator_type=None,
	serverless_inference_config=None,  # TODO: <--- add this
        tags=None,
        kms_key=None,
        data_capture_config_dict=None,
        volume_size=None,
        model_data_download_timeout=None,
        container_startup_health_check_timeout=None,
        explainer_config_dict=None,
    ):
	...

MartinJ-Dev avatar Dec 17 '23 00:12 MartinJ-Dev