synpress

synpress copied to clipboard

I'm trying to run my tests inside of GitHub Actions on an ubuntu-latest runner.

When I get to cy.confirmMetamaskSignatureRequest(), the call just times out with:

cy.task('confirmMetamaskSignatureRequest') failed with the following error:

> waiting for function failed: timeout 30000ms exceeded

https://on.cypress.io/api/task

After reviewing the screen recording in cypress, I see that the metamask popup never shows up on top of the browser.

you tried with retries?

you tried with retries?

Yes, currently making two attempts. It seems that the MetaMask popup is stuck behind the browser and Cypress or Puppeteer won't switch over to it for some reason. Are you or anyone else running in GitHub Actions? It would be nice to see an example of signing a message inside a GitHub Actions runner.

I am using on github actions but i'm not using synpress as it is being used right now. i put synpress as a plugin directly in the cypress plugins so i can use it on the cypress open command to see what happens.

There is some stuff that can happen that is not intentional like changing file spec and the metamask wallet locks so that can be a reason.

@pcardosolei can you share your GitHub Actions workflow? I'd like to see what else you're installing and how you're calling Cypress, installing the plugins, etc.

name: CI - Test
on:
  pull_request:
    branches:
      - development

# cancelling jobs automatically.
concurrency:
  group: ${{ github.head_ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    container: cypress/browsers:node16.5.0-chrome97-ff96
    env:
       VARIABLES RELATED WITH BUILD NOT RELEVANT HERE.
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - uses: actions/setup-node@v3
        with:
          node-version: "16"
          cache: "npm"
      - run: npm install
      - run: npm run build

      - name: "Tests - Chrome"
        uses: cypress-io/github-action@v2
        with:
          install: false
          headless: false
          start: npm start
          wait-on: "http://localhost:3000"
          wait-on-timeout: 120
          browser: chrome
          record: false
          parallel: false
          command: npm run cy:run
        env:
          CYPRESS_SECRET_WORDS: -
          CYPRESS_NETWORK_NAME: -
          CYPRESS_METAMASK_PW: -
          CYPRESS_SKIP_METAMASK_INSTALL: false
          CYPRESS_SKIP_METAMASK_SETUP: false
          CYPRESS_SKIP_RESOURCES_WAIT: true
          CYPRESS_REMOTE_DEBUGGING_PORT: 9222

on package.json "cy:run": "CYPRESS_REMOTE_DEBUGGING_PORT=9222 cypress run --browser chrome --headed"

Thanks for sharing. What is the purpose of CYPRESS_REMOTE_DEBUGGING_PORT? Do you ever get access to the remote Actions runner?

Are you using Synpress at all in this build?

It's a workaround that is allow cypress to access extensions I think. If you don't have it the metamask page will not open.

Synpress has it already so no need to worry about it.

What do you mean on remote actions runner.

I'm using synpress as a plugin by importing it on the commands and plugins files within cypress folder but cypress as a command.

@pcardosolei don't want to hijack the thread, but I wonder if you had any problems with loading MetaMask in GH actions? I'm using a mix of the old cypress-metamask plugin with Synpress, everything works relatively well on local environment, but in GH Puppeteer fails to find MetaMask tab within 60 seconds and I wonder whether you (or anyone else) had a similar issue?

@rarecrumb you won't see MM notifications or window on Cypress screeenshots/videos even if they are correctly rendered and interacted with. I'd add some extra logging to see what's happening (you could log it inside the node.js process via a task, otherwise I'm afraid it won't be printed)

Yes. @tofel Using the plugin it is not fast enough to load everything in time, resulting in an error ( the error is related with the bringing Metamask window to the front of the view)

Workaround: So I created a dummy test which is the first to act with a small wait to give time to puppeteer to do his thing.

In my forked Version which is a package now I'm pushing cypress and puppeteer to the latest version so I can see if it will work. Didn't test it yet.

My workaround is kinda messy because on parallel option I think it will not work. When I have significant changes I will send a PR here so synprss can also be updated if accepted.

There is some strange errors that I encounter in CI. For example reading the cookies with cypress gets a result in local but not on CI.

Hope it helps.

@pcardosolei it does help knowing I'm not alone with that issue in the CI :-) For a moment I thought it might be just performance, but then I recalled that I'm testing it on a suite of tests divided into three groups, each of them with its own beforeEach() function and all of them fail due to the same reason. Each waits 60 seconds for MetaMask to be loaded, so I'd imagine that at least the second or the third one should have some success loading the extension once in a while. So far during ~10 runs it's always failing, but I'll keep digging in.

It should be performance-related, since both the system under test and Cypress tests are dockerised, so the behaviour should be the same regardless where it runs, but...

I'll have a look at your package anyway, maybe it gives me some ideas.

Yes the more eyes on it the better. Then we can send a PR to synpress if we see that changes will benefit it

Yo @rarecrumb . Have you found any solution?

Seems like its happens to me too. Surprisingly it's an intermittent issue on me.

@rarecrumb @pcardosolei @Tofel @jakzaizzat

Hey guys, Have you tried latest @synthetixio/[email protected]? Are you using recommended docker-compose.yml setup on CI? Does this issue still persist?

Thanks, Jakub.

Yes. Where do you have an error. I might be able to help

On Fri, 27 May 2022, 20:19 Bartek Tofel, @.***> wrote:

@pcardosolei https://github.com/pcardosolei don't want to hijack the thread, but I wonder if you had any problems with loading MetaMask in GH actions? I'm using a mix of the old cypress-metamask plugin with Synpress, everything works relatively well on local environment, but in GH Puppeteer fails to find MetaMask tab within 60 seconds and I wonder whether you (or anyone else) had a similar issue?

@rarecrumb https://github.com/rarecrumb are you sure that MetaMask extension loads successfully?

— Reply to this email directly, view it on GitHub https://github.com/Synthetixio/synpress/issues/416#issuecomment-1139960319, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNFYLS4YUQAW6SLMYVQFCTVMEN2NANCNFSM5WNRBRKA . You are receiving this because you were mentioned.Message ID: @.***>

Hey,

Can you try again with @synthetixio/[email protected] and report back?

This issue should be fixed now.

Also, if you're using docker-e2e, I suggest to update to one of these:

- synthetixio/docker-e2e:14.21-debian
- synthetixio/docker-e2e:16.19-debian
- synthetixio/docker-e2e:18.13-debian

or..

- synthetixio/docker-e2e:14.21-ubuntu
- synthetixio/docker-e2e:16.19-ubuntu
- synthetixio/docker-e2e:18.13-ubuntu

Thanks, Jakub.