pytorch-lightning icon indicating copy to clipboard operation
pytorch-lightning copied to clipboard

ci: adding testing with M1 [1/2: without Fabric] [wip]

Open Borda opened this issue 1 year ago • 5 comments

What does this PR do?

bump the min mac-12 and adding mac-14 which is M1 machine ref: https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available

if test are passing, we need:

  • [ ] add new entry to group-check
Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • [x] Did you read the contributor guideline, Pull Request section?
  • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • [x] Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR. Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • [x] Is this pull request ready for review? (if not, please submit in draft mode)
  • [x] Check that all items from Before submitting are resolved
  • [x] Make sure the title is self-explanatory and the description concisely explains the PR
  • [x] Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @carmocca @borda

Borda avatar Feb 15 '24 10:02 Borda

⛈️ Required checks status: Has failure 🔴

Warning This job will need to be re-run to merge your PR. If you do not have write access to the repository, you can ask Lightning-AI/lai-frameworks to re-run it. If you push a new commit, all of CI will re-trigger.

Groups summary

🔴 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.13, oldest) success
pl-cpu (macOS-12, lightning, 3.10, 1.13) success
pl-cpu (macOS-12, lightning, 3.10, 2.1) success
pl-cpu (macOS-12, lightning, 3.10, 2.2) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13, oldest) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.2) success
pl-cpu (windows-2022, lightning, 3.8, 1.13, oldest) success
pl-cpu (windows-2022, lightning, 3.10, 1.13) success
pl-cpu (windows-2022, lightning, 3.10, 2.1) success
pl-cpu (windows-2022, lightning, 3.10, 2.2) success
pl-cpu (macOS-12, pytorch, 3.8, 1.13) success
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) failure
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success
pl-cpu (macOS-12, pytorch, 3.11, 2.1) success
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1) success
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success
pl-cpu (windows-2022, pytorch, 3.11, 2.1) success

These checks are required after the changes to .github/workflows/ci-tests-pytorch.yml.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, lightning, 3.8, 1.13, oldest) success
fabric-cpu (macOS-12, lightning, 3.10, 1.13) success
fabric-cpu (macOS-12, lightning, 3.11, 2.1) success
fabric-cpu (macOS-12, lightning, 3.11, 2.2) success
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.13, oldest) success
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1) success
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.2) success
fabric-cpu (windows-2022, lightning, 3.8, 1.13, oldest) success
fabric-cpu (windows-2022, lightning, 3.10, 1.13) success
fabric-cpu (windows-2022, lightning, 3.11, 2.1) success
fabric-cpu (windows-2022, lightning, 3.11, 2.2) success
fabric-cpu (macOS-12, fabric, 3.8, 1.13) success
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13) success
fabric-cpu (windows-2022, fabric, 3.8, 1.13) success
fabric-cpu (macOS-12, fabric, 3.11, 2.0) success
fabric-cpu (macOS-12, fabric, 3.11, 2.1) success
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0) success
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1) success
fabric-cpu (windows-2022, fabric, 3.11, 2.0) success
fabric-cpu (windows-2022, fabric, 3.11, 2.1) success

These checks are required after the changes to .github/workflows/ci-tests-fabric.yml.

🔴 lightning_app: Tests workflow
Check ID Status
app-pytest (macOS-12, lightning, 3.8, latest) success
app-pytest (macOS-12, lightning, 3.8, oldest) success
app-pytest (macOS-12, app, 3.9, latest) failure
app-pytest (macOS-12, app, 3.11, latest) success
app-pytest (ubuntu-20.04, lightning, 3.8, latest) success
app-pytest (ubuntu-20.04, lightning, 3.8, oldest) success
app-pytest (ubuntu-20.04, app, 3.9, latest) success
app-pytest (ubuntu-22.04, app, 3.11, latest) success
app-pytest (windows-2022, lightning, 3.8, latest) success
app-pytest (windows-2022, lightning, 3.8, oldest) success
app-pytest (windows-2022, app, 3.8, latest) success
app-pytest (windows-2022, app, 3.11, latest) success

These checks are required after the changes to .github/workflows/ci-tests-app.yml.

🟢 lightning_app: Examples
Check ID Status
app-examples (macOS-12, lightning, 3.9, latest) success
app-examples (macOS-12, lightning, 3.9, oldest) success
app-examples (macOS-12, app, 3.9, latest) success
app-examples (ubuntu-20.04, lightning, 3.9, latest) success
app-examples (ubuntu-20.04, lightning, 3.9, oldest) success
app-examples (ubuntu-20.04, app, 3.9, latest) success
app-examples (windows-2022, lightning, 3.9, latest) success
app-examples (windows-2022, lightning, 3.9, oldest) success
app-examples (windows-2022, app, 3.9, latest) success

These checks are required after the changes to .github/workflows/ci-examples-app.yml.

🟢 lightning: Store
Check ID Status
store-cpu (macOS-12, lightning, 3.10, 2.0) success
store-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success
store-cpu (windows-2022, lightning, 3.10, 2.0) success

These checks are required after the changes to .github/workflows/ci-tests-store.yml.


Thank you for your contribution! 💜

Note This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

github-actions[bot] avatar Feb 15 '24 10:02 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81%. Comparing base (b9680a3) to head (6385e02). Report is 122 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #19477    +/-   ##
========================================
- Coverage      83%      81%    -1%     
========================================
  Files         424      424            
  Lines       34930    34930            
========================================
- Hits        28819    28394   -425     
- Misses       6111     6536   +425     

codecov[bot] avatar Feb 15 '24 10:02 codecov[bot]

@awaelchli shall we skip these MPS tests for not enough memory or make it smaller? https://github.com/Lightning-AI/pytorch-lightning/actions/runs/8054227242/job/21998318755?pr=19477#step:14:2668

Borda avatar Feb 26 '24 21:02 Borda

Those tests don't seem too big. Maybe there's a memory leak making later tests fail. It will need some research

carmocca avatar Feb 26 '24 22:02 carmocca

lets skip Fabric for now and enable for the rest

Borda avatar Mar 07 '24 13:03 Borda