pytorch-lightning icon indicating copy to clipboard operation
pytorch-lightning copied to clipboard

Convert subprocess test to standalone test

Open awaelchli opened this issue 3 years ago • 1 comments

What does this PR do?

Needed for #13967

In #13967 there are two tests that fail:

  • test_multi_gpu_model_ddp_fit_only
  • test_multi_gpu_model_ddp_fit_test

These tests are unrelated to deepspeed, yet they fail consistently when I upgrade deepspeed in the CI. It is not reproducible locally and I couldn't figure out the reason. However, these tests are old, and if I convert them to standalone tests like we have for other similar ddp tests, there is no problem.

Hence, I propose to just refactor these old tests into standalone tests to move forward. It would have been done sooner or later anyway.

Before submitting

  • [x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • [x] Did you read the contributor guideline, Pull Request section?
  • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
  • [x] Did you make sure to update the documentation with your changes? (if necessary)
  • [x] Did you write any new necessary tests? (not for typos and docs)
  • [x] Did you verify new and existing tests pass locally with your changes?
  • [x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed. Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • [x] Is this pull request ready for review? (if not, please submit in draft mode)
  • [x] Check that all items from Before submitting are resolved
  • [x] Make sure the title is self-explanatory and the description concisely explains the PR
  • [x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

I made sure I had fun coding 🙃

cc @justusschock @awaelchli @rohitgr7 @borda @akihironitta @kaushikb11

awaelchli avatar Aug 08 '22 21:08 awaelchli

Codecov Report

Merging #14101 (9ce4cbf) into master (dc8ff5e) will increase coverage by 15%. The diff coverage is n/a.

:exclamation: Current head 9ce4cbf differs from pull request most recent head 25e4f11. Consider uploading reports for the commit 25e4f11 to get more accurate results

@@            Coverage Diff            @@
##           master   #14101     +/-   ##
=========================================
+ Coverage      61%      76%    +15%     
=========================================
  Files         324      326      +2     
  Lines       26408    26572    +164     
=========================================
+ Hits        16144    20227   +4083     
+ Misses      10264     6345   -3919     

codecov[bot] avatar Aug 08 '22 23:08 codecov[bot]

The test failures here happen on master too, so I'm not going to investigate in this PR.

test_configure_external_callbacks

 TypeError: 'Mock' object is not iterable

awaelchli avatar Aug 10 '22 21:08 awaelchli