deep-learning-containers icon indicating copy to clipboard operation
deep-learning-containers copied to clipboard
verified publisher icon aws

[pytorch][sagemaker]Port https://github.com/aws/deep-learning-containers/pull/3293 to Cu121

Open emeraldbay opened this issue 1 year ago • 4 comments

GitHub Issue #, if available:

Note:

  • If merging this PR should also close the associated Issue, please also add that Issue # to the Linked Issues section on the right.

  • All PR's are checked weekly for staleness. This PR will be closed if not updated in 30 days.

Description

Port https://github.com/aws/deep-learning-containers/pull/3293 to Cu121/Pytorch 2.0

Tests run

NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"

  • [X] I have run builds/tests on commit 28a518dc75ea496db55c27baa2337124617562e5 for my changes.

NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:

  • [ ] Revision A: sagemaker_remote_tests = "standard"
  • [ ] Revision B: sagemaker_remote_tests = "rc"
  • [ ] Revision C: sagemaker_remote_tests = "efa"

Additionally, please run the sagemaker local tests in at least one revision:

  • [ ] sagemaker_local_tests = true

Formatting

  • [X] I have run black -l 100 on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)

DLC image/dockerfile

Additional context

PR Checklist

  • [X] I've prepended PR tag with frameworks/job this applies to : [mxnet, tensorflow, pytorch] | [ei/neuron/graviton] | [build] | [test] | [benchmark] | [ec2, ecs, eks, sagemaker]
  • [X] If the PR changes affects SM test, I've modified dlc_developer_config.toml in my PR branch by setting sagemaker_tests = true and efa_tests = true
  • [X] If this PR changes existing code, the change fully backward compatible with pre-existing code. (Non backward-compatible changes need special approval.)
  • [X] (If applicable) I've documented below the DLC image/dockerfile this relates to
  • [X] (If applicable) I've documented below the tests I've run on the DLC image
  • [X] (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list. See https://www.apache.org/legal/resolved.html.
  • [X] (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them.

Pytest Marker Checklist

  • [ ] (If applicable) I have added the marker @pytest.mark.model("<model-type>") to the new tests which I have added, to specify the Deep Learning model that is used in the test (use "N/A" if the test doesn't use a model)
  • [ ] (If applicable) I have added the marker @pytest.mark.integration("<feature-being-tested>") to the new tests which I have added, to specify the feature that will be tested
  • [ ] (If applicable) I have added the marker @pytest.mark.multinode(<integer-num-nodes>) to the new tests which I have added, to specify the number of nodes used on a multi-node test
  • [ ] (If applicable) I have added the marker @pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">) to the new tests which I have added, if a test is specifically applicable to only one processor type

EIA/NEURON/GRAVITON Testing Checklist

  • When creating a PR:
  • [ ] I've modified dlc_developer_config.toml in my PR branch by setting ei_mode = true, neuron_mode = true or graviton_mode = true

Benchmark Testing Checklist

  • When creating a PR:
  • [ ] I've modified dlc_developer_config.toml in my PR branch by setting benchmark_mode = true

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

emeraldbay avatar Jan 08 '24 22:01 emeraldbay

I guess we are not running the tests from #3293 on this PR. We would need to do that to ensure package has been installed properly.

I have enabled these tests in Cu121 build

emeraldbay avatar Feb 06 '24 00:02 emeraldbay

This PR has been marked stale as a result of being open for 30 days without activity or updates. Please remove the stale label or comment in order to keep this open, otherwise the PR will be closed in 5 days.

github-actions[bot] avatar Mar 31 '24 19:03 github-actions[bot]

Hi, what is latest for this PR?

emeraldbay avatar Mar 31 '24 19:03 emeraldbay