WIP-DEBUG-PROFILE torch.compile

[IvanKobzarev]

Stack from ghstack (oldest at bottom):

• -> #2644

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/2644

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 4 New Failures, 3 Cancelled Jobs

As of commit c3f6a7dc4e76d170ff041cf0521e14b12b202ade with merge base 0991f97ef13735ea6d458db22137a2796f8fbf92 ():

NEW FAILURES - The following jobs have failed:

• GPU tests / gpu_test (3.10, stable) (gh) tests/torchtune/modules/moe/test_moe.py::TestMoeLayer::test_forward
• GPU tests / gpu_test (3.11, stable) (gh) tests/torchtune/modules/moe/test_moe.py::TestMoeLayer::test_forward
• Lint / lint (3.10) (gh) torchtune/training/_profiler.py:88:5: F811 redefinition of unused 'time' from line 10
• Unit Test / unit_tests (3.10) (gh) tests/torchtune/modules/moe/test_experts.py::TestLoRAGroupedExperts::test_forward_disabled

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• GPU tests / gpu_test (3.9, stable) (gh)
• Unit Test / unit_tests (3.11) (gh) ##[error]The operation was canceled.
• Unit Test / unit_tests (3.9) (gh) ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 15.23810% with 89 lines in your changes missing coverage. Please review.

Please upload report for BASE (gh/IvanKobzarev/3/base@43b72a4). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
torchtune/modules/moe/indices.py	0.00%	68 Missing :warning:
torchtune/modules/moe/experts.py	56.52%	10 Missing :warning:
torchtune/modules/moe/moe.py	27.27%	8 Missing :warning:
torchtune/training/_profiler.py	0.00%	3 Missing :warning:

Additional details and impacted files

@@                    Coverage Diff                    @@
##             gh/IvanKobzarev/3/base    #2644   +/-   ##
=========================================================
  Coverage                          ?   66.39%           
=========================================================
  Files                             ?      400           
  Lines                             ?    24374           
  Branches                          ?        0           
=========================================================
  Hits                              ?    16182           
  Misses                            ?     8192           
  Partials                          ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.