SageMaker deployment of a HuggingFace model in local mode fails with KeyError: 'ModelDataUrl'

[artmatsak]

Describe the bug SageMaker deployment of a HuggingFace model from the Hub in local mode fails with KeyError: 'ModelDataUrl' in entities.py.

To reproduce

1. Open the example HuggingFace deployment notebook over here.
2. Set instance_type to "local" for deployment:

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
   initial_instance_count=1,
   instance_type="local"
)

3. Run the notebook.
4. The deployment will fail with the following error:

.../venv/lib/python3.8/site-packages/sagemaker/local/entities.py in serve(self)
    576         )
    577         self.container.serve(
--> 578             self.primary_container["ModelDataUrl"], self.primary_container["Environment"]
    579         )
    580 

KeyError: 'ModelDataUrl'

Expected behavior The model is deployed locally.

System information

• SageMaker Python SDK version: 2.66.1
• Framework name (eg. PyTorch) or algorithm (eg. KMeans): HuggingFace Transformers
• Framework version: 4.11.3
• Python version: 3.8.10
• CPU or GPU: N/A
• Custom Docker image (Y/N): N

Additional context Originally reported here: https://github.com/aws/sagemaker-huggingface-inference-toolkit/issues/35

My assumption is that the bug is caused by model_data not being passed to the HuggingFaceModel constructor and us relying on env instead.

[timxieICN]

Hi I'm running into the same issue.

[timxieICN]

OK - looks like the workaround is:

model = HuggingFaceModel(
            # env=hub,   # Do not use hub
            model_data="s3://xxxxx/local-test-model/model.tar.gz",