amazon-sagemaker-examples Add Bloom 560m parameter model example using SageMaker Model Parallel…

… with Sharded Data Parallelism through a custom SMP Trainer.

This example shows you how to use SMP Trainer as a drop-in replacement for HuggingFace Trainer to enable Sharded Data Parallelism with SageMaker Model Parallel.

Testing done: Ran the notebook, and tested loading from partial and full checkpoints.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

[x] I have read the CONTRIBUTING doc and adhered to the example notebook best practices
[ ] I have updated any necessary documentation, including READMEs
[x] I have tested my notebook(s) and ensured it runs end-to-end
[x] I have linted my notebook(s) and code using black-nb -l 100 {path}/{notebook-name}.ipynb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Jun 02 '23 23:06 viclzhu

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Jun 02 '23 23:06 review-notebook-app[bot]

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
Commit ID: 350bcbc6b8bd7e9d9cb629c09ba97ab4056fe9cc
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 02 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
Commit ID: 350bcbc6b8bd7e9d9cb629c09ba97ab4056fe9cc
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 02 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-grammar
Commit ID: 350bcbc6b8bd7e9d9cb629c09ba97ab4056fe9cc
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 02 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
Commit ID: 350bcbc6b8bd7e9d9cb629c09ba97ab4056fe9cc
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 02 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
Commit ID: d8552bd66d64552803f680b3cc4231703bcabb94
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 20:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
Commit ID: d8552bd66d64552803f680b3cc4231703bcabb94
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 20:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-grammar
Commit ID: d8552bd66d64552803f680b3cc4231703bcabb94
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 20:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
Commit ID: d8552bd66d64552803f680b3cc4231703bcabb94
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 20:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
Commit ID: bbe48769ddbdcdfdc89721393bceea4cd7953fc3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 22:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
Commit ID: bbe48769ddbdcdfdc89721393bceea4cd7953fc3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 22:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-grammar
Commit ID: bbe48769ddbdcdfdc89721393bceea4cd7953fc3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 22:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
Commit ID: bbe48769ddbdcdfdc89721393bceea4cd7953fc3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
Commit ID: babee73b13ed4fbc8b7fed9092a4c53a7aab243c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 20 '23 23:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
Commit ID: babee73b13ed4fbc8b7fed9092a4c53a7aab243c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 21 '23 00:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-grammar
Commit ID: babee73b13ed4fbc8b7fed9092a4c53a7aab243c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 21 '23 00:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
Commit ID: babee73b13ed4fbc8b7fed9092a4c53a7aab243c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 21 '23 00:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
Commit ID: 1eb3b1b5575bee5943531e56a87a3260dfe6221e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 26 '23 18:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-grammar
Commit ID: 1eb3b1b5575bee5943531e56a87a3260dfe6221e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 26 '23 18:06 sagemaker-bot

AWS CodeBuild CI Report

CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
Commit ID: 1eb3b1b5575bee5943531e56a87a3260dfe6221e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Jun 26 '23 18:06 sagemaker-bot