pytorch-lightning icon indicating copy to clipboard operation
pytorch-lightning copied to clipboard

Delay `Precision.convert_module` until `configure_model` has run

Open carmocca opened this issue 2 years ago • 3 comments

What does this PR do?

Fixes #18936

The main change is in src/lightning/pytorch/strategies/strategy.py. Everything else is a bit of flattening and making it as consistent as possible with its subclasses.

The half precision test is redundant. It wasn't failing before this PR. The relevant tests are the TransformerEngine and Bitsandbytes ones.


:books: Documentation preview :books:: https://pytorch-lightning--19061.org.readthedocs.build/en/19061/

cc @carmocca @justusschock @awaelchli

carmocca avatar Nov 23 '23 19:11 carmocca

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.13, oldest) success
pl-cpu (macOS-11, lightning, 3.10, 1.13) success
pl-cpu (macOS-11, lightning, 3.10, 2.1) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13, oldest) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success
pl-cpu (windows-2022, lightning, 3.8, 1.13, oldest) success
pl-cpu (windows-2022, lightning, 3.10, 1.13) success
pl-cpu (windows-2022, lightning, 3.10, 2.1) success
pl-cpu (macOS-11, pytorch, 3.8, 1.13) success
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) success
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success
pl-cpu (macOS-12, pytorch, 3.11, 2.1) success
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1) success
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success
pl-cpu (windows-2022, pytorch, 3.11, 2.1) success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py, tests/tests_pytorch/plugins/precision/test_bitsandbytes.py, tests/tests_pytorch/plugins/precision/test_half.py, tests/tests_pytorch/plugins/precision/test_transformer_engine.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) (testing Lightning | latest) success
pytorch-lightning (GPUs) (testing PyTorch | latest) success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py, tests/tests_pytorch/plugins/precision/test_bitsandbytes.py, tests/tests_pytorch/plugins/precision/test_half.py, tests/tests_pytorch/plugins/precision/test_transformer_engine.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
lightning.Benchmarks success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py.

🟢 pytorch_lightning: Docs
Check ID Status
docs-make (pytorch, doctest) success
docs-make (pytorch, html) success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py.

🟢 mypy
Check ID Status
mypy success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success
install-pkg (ubuntu-22.04, app, 3.11) success
install-pkg (ubuntu-22.04, fabric, 3.8) success
install-pkg (ubuntu-22.04, fabric, 3.11) success
install-pkg (ubuntu-22.04, pytorch, 3.8) success
install-pkg (ubuntu-22.04, pytorch, 3.11) success
install-pkg (ubuntu-22.04, lightning, 3.8) success
install-pkg (ubuntu-22.04, lightning, 3.11) success
install-pkg (ubuntu-22.04, notset, 3.8) success
install-pkg (ubuntu-22.04, notset, 3.11) success
install-pkg (macOS-12, app, 3.8) success
install-pkg (macOS-12, app, 3.11) success
install-pkg (macOS-12, fabric, 3.8) success
install-pkg (macOS-12, fabric, 3.11) success
install-pkg (macOS-12, pytorch, 3.8) success
install-pkg (macOS-12, pytorch, 3.11) success
install-pkg (macOS-12, lightning, 3.8) success
install-pkg (macOS-12, lightning, 3.11) success
install-pkg (macOS-12, notset, 3.8) success
install-pkg (macOS-12, notset, 3.11) success
install-pkg (windows-2022, app, 3.8) success
install-pkg (windows-2022, app, 3.11) success
install-pkg (windows-2022, fabric, 3.8) success
install-pkg (windows-2022, fabric, 3.11) success
install-pkg (windows-2022, pytorch, 3.8) success
install-pkg (windows-2022, pytorch, 3.11) success
install-pkg (windows-2022, lightning, 3.8) success
install-pkg (windows-2022, lightning, 3.11) success
install-pkg (windows-2022, notset, 3.8) success
install-pkg (windows-2022, notset, 3.11) success

These checks are required after the changes to src/lightning/pytorch/strategies/ddp.py, src/lightning/pytorch/strategies/deepspeed.py, src/lightning/pytorch/strategies/fsdp.py, src/lightning/pytorch/strategies/single_device.py, src/lightning/pytorch/strategies/single_xla.py, src/lightning/pytorch/strategies/strategy.py, src/lightning/pytorch/strategies/xla.py.


Thank you for your contribution! 💜

Note This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

github-actions[bot] avatar Nov 23 '23 20:11 github-actions[bot]

Codecov Report

Merging #19061 (24a1b52) into master (c7c42dc) will decrease coverage by 33%. Report is 11 commits behind head on master. The diff coverage is 83%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19061      +/-   ##
==========================================
- Coverage      82%      49%     -33%     
==========================================
  Files         448      440       -8     
  Lines       37915    37783     -132     
==========================================
- Hits        30985    18467   -12518     
- Misses       6930    19316   +12386     

codecov[bot] avatar Nov 23 '23 20:11 codecov[bot]

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them. Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately. Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!

gitguardian[bot] avatar Jan 16 '24 09:01 gitguardian[bot]

The failing TPU job is for Fabric only, but Fabric isn't impacted by this PR so it must be an issue from somewhere else

carmocca avatar Feb 05 '24 23:02 carmocca