metamask-extension icon indicating copy to clipboard operation
metamask-extension copied to clipboard

test/e2e: allow overriding some browser settings via env vars

Open legobeat opened this issue 1 year ago • 3 comments

Description

Allow configuring some browser behavior for local e2e testing using the following environment variables:

  • Override https proxy: SELENIUM_HTTPS_PROXY
  • Disable DNS-over-TLS for chromium (using system DNS): SELENIUM_DISABLE_DOT
  • Enable headless mode: SELENIUM_HEADLESS

Also disables some unnecessary phoning-home to Google servers by chromium during testing.

Related issues

Fixes:

Manual testing steps

  1. Create test build: yarn build:test
  2. Start a local http proxy listening on e.g. 127.0.0.1:1234
  3. chromium: Initiate headless e2e test with DoT disabled SELENIUM_HTTPS_PROXY=http://127.0.0.1:1234 SELENIUM_DISABLE_DOT=1 SELENIUM_HEADLESS=1 yarn test:e2e:chrome --build-type main
  • observe queries hitting your proxy, tests executing with no browser window opening, and no DoT requests being made
  1. firefox: Initiate headless e2e test: SELENIUM_HTTPS_PROXY=http://127.0.0.1:1234 SELENIUM_HEADLESS=1 yarn test:e2e:firefox --build-type main
  • observe queries hitting your proxy, tests executing with no browser window opening

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • [x] I’ve followed MetaMask Coding Standards.
  • [x] I've clearly explained what problem this PR is solving and how it is solved.
  • [x] I've linked related issues
  • [x] I've included manual testing steps
  • [x] I've included screenshots/recordings if applicable
  • [x] I’ve included tests if applicable
  • [x] I’ve documented my code using JSDoc format if applicable
  • [x] I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
  • [x] I’ve properly set the pull request status:
    • [x] In case it's not yet "ready for review", I've set it to "draft".
    • [x] In case it's "ready for review", I've changed it from "draft" to "non-draft".

Pre-merge reviewer checklist

  • [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

legobeat avatar Feb 08 '24 15:02 legobeat

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

github-actions[bot] avatar Feb 08 '24 15:02 github-actions[bot]

This is very very interesting, but I have a few questions:

  • Under what circumstances would you want to override the included proxy at 127.0.0.1:8000?
  • SELENIUM_HEADLESS fails on a lot of our tests, it would be nice to know why (this is not necessarily a question to Lego). Then we could maybe run headless on CircleCI.

HowardBraham avatar Feb 08 '24 21:02 HowardBraham

This is very very interesting, but I have a few questions:

* Under what circumstances would you want to override the included proxy at `127.0.0.1:8000`?

Might want to do some auditing, debugging, interception or what not with http calls being made. For example a local squid or mitmproxy instance. There are probably use-cases I didn't think of too ^^

* `SELENIUM_HEADLESS` fails on a lot of our tests, it would be nice to know why (this is not necessarily a question to Lego).  Then we could maybe run headless on CircleCI.

So it's not just me then...

But I have some consistent e2e test failures that happen locally on develop without any of these overrides and figuring these help in debugging situations like that locally. At this point I haven't figured out yet where the discrepancy in results locally vs circleci comes from.

legobeat avatar Feb 08 '24 22:02 legobeat

No release label on PR. Adding release label release-11.12.0 on PR, as PR was added to branch 11.12.0 when release was cut.

metamaskbot avatar Mar 01 '24 18:03 metamaskbot