Question: train and deploy via Sagemaker

[lbelpaire]

Hi

I'm trying to setup training (and hyperparameter tuning) using Amazon SageMaker. Because SetFit is not a standard model on HugginFace I'm guessing that the examples provided in the HuggingFace/SageMaker integration are not useable: example.

What would the best way to tackle hyperparameter tuning (tuning body and head separately) on SageMaker and track the results?

[tomaarsen]

I don't have any experience with Sagemaker, so I can't really help out here, other than to say that the examples are indeed not applicable I'm afraid.

• Tom Aarsen

[aaronvg]

I got it working by following the huggingface library docs for sage maker deployments. They have a way to deploy any Hf model from s3. All you need is a custom inference.py script as the docs suggest. Sorry i dont have the link handy at the moment.

[miteshkotak]

@aaronvg I am trying to deploy using sagemaker pipeline for model deployment did you use huggingface_model.deploy or ModelDeployment ? when i try to deploy the setfit model using "huggingface_model.deploy" in sagemaker pipeline.

i get the following error `File /opt/conda/lib/python3.10/site-packages/sagemaker/workflow/pipeline_context.py:190, in PipelineSession._intercept_create_request(self, request, create, func_name) 182 """This function intercepts the create job request 183 184 Args: (...) 187 func_name (str): the name of the function needed intercepting 188 """ 189 if func_name == self.create_model.name: --> 190 self.context.create_model_request = request 191 self.context.caller_name = func_name 192 elif func_name == self.create_model_package_from_containers.name:

AttributeError: 'NoneType' object has no attribute 'create_model_request'`

any idea? how to solve this ?

i get the following error

`File /opt/conda/lib/python3.10/site-packages/sagemaker/workflow/pipeline_context.py:190, in PipelineSession._intercept_create_request(self, request, create, func_name)
    182 """This function intercepts the create job request
    183 
    184 Args:
   (...)
    187     func_name (str): the name of the function needed intercepting
    188 """
    189 if func_name == self.create_model.__name__:
--> 190     self.context.create_model_request = request
    191     self.context.caller_name = func_name
    192 elif func_name == self.create_model_package_from_containers.__name__:

AttributeError: 'NoneType' object has no attribute 'create_model_request'`

any idea? how to solve this ?

[miteshkotak]

I got it resolved. was using the wrong method to deploy (hf.deploy) instead of custom deployment class

[tomaarsen]

I'm glad you got it resolved! If you're open to it, then feel free to share your custom inference.py script & deployment steps. I'm looking to improve the documentation, and that way I can add try to it for others who are interested in deploying via Sagemaker.

[miteshkotak]

@tomaarsen absolutely i will share custom inference script. i will need some more time for sage maker pipeline script to make it perfect

[ratkins]

Any hints on how to perform training on Sagemaker, not inference? We want to follow the tutorial but train in the cloud.