generative-ai-cdk-constructs icon indicating copy to clipboard operation
generative-ai-cdk-constructs copied to clipboard

support custom code for Hugging Face SageMaker model deployment

Open krokoko opened this issue 1 year ago • 0 comments

Describe the feature

In some cases, the user needs to provide custom logic to deploy a HF model to a SG endpoint:

  • overwrite/add dependencies to the existing DLC
  • add custom inference script to manage inference requests In those cases, the user needs to use a notebook instance, SG Studio or a local environment to This step could be managed by the construct

Use Case

Avoid context switching and deploy more models directly through CDK

Proposed Solution

Add a new optional property customCode pointing to a location (local or S3) containing the custom logic of the user. If provided, the construct would run a CodeBuild step to:

  • pull model artifacts from HF to an S3 location
  • add user's custom logic
  • deploy the model from the new S3 location

Other Information

No response

Acknowledgements

  • [ ] I may be able to implement this feature request
  • [ ] This feature might incur a breaking change

krokoko avatar Mar 11 '24 20:03 krokoko