metamask-extension icon indicating copy to clipboard operation
metamask-extension copied to clipboard

feat: improved way to trigger mmi e2e tests

Open albertolive opened this issue 4 months ago • 6 comments

Description

Summary

This pull request introduces a conditional check in the CircleCI pipeline to determine whether MMI Playwright tests should be executed based on the presence of the team-mmi label or if a reviewer from the MetaMask/mmi team has been assigned to the pull request. This optimizes the CI workflow by running MMI-specific tests only when necessary.

Previously, the end-to-end tests were only triggered when the team-mmi label was added to a pull request. However, in practice, no one outside of our team was adding this label, which resulted in the end-to-end MMI tests never being executed.

We already have an automation in place to automatically add a reviewer from the MetaMask/mmi team when relevant pull requests are opened. This pull request leverages that automation to ensure that the MMI tests are automatically triggered when a reviewer is assigned, making the process more efficient and reducing manual steps.

Changes Implemented

  1. MMI Trigger Check
  • Added a check-mmi-trigger step that runs a shell script to check if the pull request has either:
    • A team-mmi label.
    • A reviewer from the MetaMask/mmi team assigned.
  • If either condition is met, the MMI-specific tests will be triggered. Otherwise, the workflow halts and skips these tests.
  1. Modified Workflow
  • The prep-build-test-mmi-playwright and test-e2e-mmi-playwright steps are now conditionally executed based on the result of the check-mmi-trigger step.
  • If the MMI trigger is not present, the workflow halts after the check-mmi-trigger step, improving pipeline efficiency by avoiding unnecessary test execution.
  1. Shell Script (check_mmi_trigger.sh)
  • Added a script to check for the presence of the team-mmi label or the MetaMask/mmi reviewer in a pull request.
  • The script sets the run_mmi_tests environment variable to true or false depending on the conditions, which controls whether the MMI tests will proceed.

Benefits

  • Increased efficiency: MMI Playwright tests are run only when necessary, reducing CI resource usage and speeding up the overall workflow.
  • Clearer conditions for MMI tests: The conditions for triggering MMI-specific tests are now based on both labels and reviewer assignments, ensuring greater flexibility and accuracy in when the tests are executed.

Related issues

Fixes:

Manual testing steps

  1. Go to the CircleCI Project Page:
  • Navigate to the project pipeline in CircleCI.
  1. Trigger a Workflow:
  • Manually trigger the workflow by running a new build. You can either rerun a previous build or push a new commit to the branch you’re testing.
  1. Monitor the Workflow:
  • Observe the pipeline’s progress and ensure that the correct jobs (like MMI Playwright tests) are triggered based on your implemented conditions.
  • Check that the check-mmi-trigger step is evaluated correctly and logs the expected conditions met or skipped.
  1. Verify Expected Behavior:
  • If the conditions for MMI tests (label/reviewer) are met, verify that the prep-build-test-mmi-playwright and test-e2e-mmi-playwright steps run successfully.
  • If conditions are not met, ensure the MMI tests are skipped as expected.
  1. Check Logs:
  • Review the logs for each step to confirm that the correct conditions triggered the tests or that they were correctly skipped.

Screenshots/Recordings

Before

image

After

image image

Pre-merge author checklist

Pre-merge reviewer checklist

  • [x] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • [x] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

albertolive avatar Oct 17 '24 13:10 albertolive