[DTensor] Turn on foreach implementation of optimizer for DTensor by default

[wz337]

Append DTensor to the optimizer _foreach_supported_types and turn on foreach implementation of optimizer for DTensor if not specified by the users.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @tianyu-l @wconstab @yf225 @chauhang

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/123394

• :page_facing_up: Preview Python docs built from this PR
• :page_facing_up: Preview C++ docs built from this PR
• :question: Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: You can merge normally! (2 Unrelated Failures)

As of commit f907706e7d2c9e120fb7265e4ece80d54f45786a with merge base 1a28f731dc6a1d7db04518a74858f0282b686375 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• periodic / linux-focal-rocm6.1-py3.8 / test (distributed, 2, 2, linux.rocm.gpu) (gh) (disabled by #126296) distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_custom_transformer
• periodic / linux-vulkan-focal-py3.11-clang10 / test (default, 1, 1, linux.2xlarge) (gh) (similar failure) Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[wz337]

@pytorchmergebot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased turn_on_dtensor_foreach onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout turn_on_dtensor_foreach && git pull --rebase)

[wz337]

@pytorchmergebot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased turn_on_dtensor_foreach onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout turn_on_dtensor_foreach && git pull --rebase)

[wz337]

@pytorchmergebot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased turn_on_dtensor_foreach onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout turn_on_dtensor_foreach && git pull --rebase)

[wz337]

@pytorchmergebot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here

[wz337]

@pytorchmergebot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status here