metamask-extension
metamask-extension copied to clipboard
feat: improved way to trigger mmi e2e tests
Description
Summary
This pull request introduces a conditional check in the CircleCI pipeline to determine whether MMI Playwright tests should be executed based on the presence of the team-mmi
label or if a reviewer from the MetaMask/mmi
team has been assigned to the pull request. This optimizes the CI workflow by running MMI-specific tests only when necessary.
Previously, the end-to-end tests were only triggered when the team-mmi
label was added to a pull request. However, in practice, no one outside of our team was adding this label, which resulted in the end-to-end MMI tests never being executed.
We already have an automation in place to automatically add a reviewer from the MetaMask/mmi
team when relevant pull requests are opened. This pull request leverages that automation to ensure that the MMI tests are automatically triggered when a reviewer is assigned, making the process more efficient and reducing manual steps.
Changes Implemented
- MMI Trigger Check
- Added a
check-mmi-trigger
step that runs a shell script to check if the pull request has either:- A
team-mmi
label. - A reviewer from the
MetaMask/mmi
team assigned.
- A
- If either condition is met, the MMI-specific tests will be triggered. Otherwise, the workflow halts and skips these tests.
- Modified Workflow
- The
prep-build-test-mmi-playwright
andtest-e2e-mmi-playwright
steps are now conditionally executed based on the result of thecheck-mmi-trigger
step. - If the MMI trigger is not present, the workflow halts after the
check-mmi-trigger
step, improving pipeline efficiency by avoiding unnecessary test execution.
- Shell Script (check_mmi_trigger.sh)
- Added a script to check for the presence of the
team-mmi
label or theMetaMask/mmi
reviewer in a pull request. - The script sets the
run_mmi_tests
environment variable to true or false depending on the conditions, which controls whether the MMI tests will proceed.
Benefits
- Increased efficiency: MMI Playwright tests are run only when necessary, reducing CI resource usage and speeding up the overall workflow.
- Clearer conditions for MMI tests: The conditions for triggering MMI-specific tests are now based on both labels and reviewer assignments, ensuring greater flexibility and accuracy in when the tests are executed.
Related issues
Fixes:
Manual testing steps
- Go to the CircleCI Project Page:
- Navigate to the project pipeline in CircleCI.
- Trigger a Workflow:
- Manually trigger the workflow by running a new build. You can either rerun a previous build or push a new commit to the branch you’re testing.
- Monitor the Workflow:
- Observe the pipeline’s progress and ensure that the correct jobs (like MMI Playwright tests) are triggered based on your implemented conditions.
- Check that the
check-mmi-trigger
step is evaluated correctly and logs the expected conditions met or skipped.
- Verify Expected Behavior:
- If the conditions for MMI tests (label/reviewer) are met, verify that the
prep-build-test-mmi-playwright
andtest-e2e-mmi-playwright
steps run successfully. - If conditions are not met, ensure the MMI tests are skipped as expected.
- Check Logs:
- Review the logs for each step to confirm that the correct conditions triggered the tests or that they were correctly skipped.
Screenshots/Recordings
Before
After
Pre-merge author checklist
- [x] I've followed MetaMask Contributor Docs and MetaMask Extension Coding Standards.
- [x] I've completed the PR template to the best of my ability
- [x] I’ve included tests if applicable
- [x] I’ve documented my code using JSDoc format if applicable
- [x] I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
Pre-merge reviewer checklist
- [x] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
- [x] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.