Update Cypress to 10.10.0

[MikeMcC399]

This PR implements the enhancement request https://github.com/corona-warn-app/cwa-website/issues/3100 "Update Cypress to latest version (10)" to update Cypress to version 10.10.0, which is currently the latest available version.

A side effect of this update is to allow Cypress to support the the Apple M1 ARM architecture natively without requiring Rosetta to run.

See Cypress Changelog.

Impact

Cypress 10 introduces breaking changes due to the new functionality of component testing which is added alongside the existing end-to-end testing (e2e). For instance, new folders are introduced, so that the existing cypress/integration folder is migrated to cypress/e2e. Also test specs are renamed: app_to_web.js becomes app_to_web.cy.js. See the Cypress Migration Guide for more detail.

In Cypress 10 the "Run all specs" is no longer available in the UI. Only individual tests can be run from the UI. To run all tests the CLI needs to be used. See Discussion: 'Run all specs' removal in Cypress 10.

Migration steps

The following steps have been carried out by this PR:

Install Cypress 10

1. npm install -D cypress@latest (installs [email protected] - see Changelog)
2. npx cypress open
3. Click "Continue to Cypress 10"

Migration tool

In the migration tool which automatically starts in the Cypress UI:

1. Click "Rename these specs for me"
2. Click "Rename the support file for me"
3. Click "Migrate the configuration for me"

• "E2E Testing" is now marked as "Configured"
• "Component Testing" is left as "Not Configured"

4. Close Cypress test runner window

Post installation

1. Edit .github/workflows/cypress-test-prod.yml and change cypress spec folder from integration to e2e i.e. spec: cypress/e2e/*.js
2. Modify the cypress:open script in package.json to use e2e and browser Chrome i.e. "cypress:open": "cypress open --e2e --browser chrome"
3. Update README.md regarding change of folder and removal of "all tests" option in open command.
4. Add cypress/downloads/ to .gitignore

Verification

After fetching the branch MikeMcC399:cypress-10 execute npm ci to install Cypress 10.10.0.

Execute the following and check that all tests pass:

npm run test:chrome
npm run test:firefox
npm run test
npx cypress run -s 'cypress/e2e/*.js' -c baseUrl=https://coronawarn.app --headless --browser chrome

Execute npm run test:open and check that the list of e2e test specs is shown immediately and the default browser is Chrome. Select and run one of the tests listed, such as mime.cy.js.

Test on:

• Windows 11
• Ubuntu 20.04
• macOS

Note that there is currently an issue with a warning message "Couldn't find tsconfig.json. tsconfig-paths will be skipped". See issue 22273 in https://github.com/cypress-io/cypress/issues/22273. This warning can be ignored.

Test the workflow .github/workflows/cypress-test-prod.yml and ensure that it passes.

---

Internal Tracking ID: EXPOSUREAPP-14080

[MikeMcC399]

Windows 11 tests

• [x] npx cypress run -s 'cypress/e2e/*.js' -c baseUrl=https://coronawarn.app --headless --browser chrome
• [ ] npm run test:chrome
• [ ] npm run test:firefox
• [ ] npm run test

[MikeMcC399]

Ubuntu tests

Ubuntu 20.04.5 LTS running under WSL 2 (Windows Subsystem for Linux 2)

• [x] npx cypress run -s 'cypress/e2e/*.js' --headless --browser chrome running in a separate terminal window to npm start

native Ubuntu 20.04.5 LTS

• [ ] npm run test:chrome
• [ ] npm run test:firefox
• [ ] npm run test

[brifemu]

Hey @MikeMcC399, it's a pleasure to be back in touch with you after so long. Hope everything is going well for you.

I've tested your PR on Mac, exactly this one:

Everything works perfectly, you have done a great job. Thanks a lot! I am preparing tests on Windows 10 and Ubuntu, I will write a new comment when they are finished.

Once again, nice to be back in touch with you!

[MikeMcC399]

@brifemu

Well it's good to hear from you again as well! Thank you for testing on Mac.

I did a brief successful re-test on Windows and Ubuntu. I will remove the Draft status on this PR now.

[Ein-Tim]

Please also consider testing this PR on a Mac(Book) using an Apple silicon (M1 / M2) processor & with the latest macOS (macOS Monterey 12.6).

cc @brifemu

[brifemu]

Tested on Windows 10 Enterprise 20H2 build 19042.1706 and working!

[MikeMcC399]

@mtb77

Thank you for running M1 tests!

The applink.cy.js test is failing, whereas the other tests are successful:

│ ✖ applink.cy.js 00:13 3 - 3 - - │

Could you try running the test individually to see what the problem is?

npm run test:open then select applink.cy.js

[larswmh]

@MikeMcC399

The applink.cy.js test is failing, whereas the other tests are successful:

│ ✖ applink.cy.js 00:13 3 - 3 - - │

Could you try running the test individually to see what the problem is?

Tested this on my MacBook Pro 14" 2021, M1 Pro and did not run into the error. Full test shown below. For the other two failing tests, they are most likely related to the gulp-imagemin package, where I had to temporarily remove the corresponding code in order to run the site locally. For some reason, it failed on .jpeg and .jpg files.

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  app_to_web.cy.js                         00:06        8        8        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  applink.cy.js                            425ms        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  blog.cy.js                               439ms        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  check_anchor_links.cy.js                 00:33       37       37        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  check_videos.cy.js                       00:16       39       39        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  eventRegistration.cy.js                  00:14        6        4        2        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  faq.cy.js                                00:09       16       16        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  hotline.cy.js                            252ms        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  mime.cy.js                               125ms        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  hybrid/check_links.cy.js                 13:47       39       35        4        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  2 of 10 failed (20%)                     15:08      152      146        6        -        -  

Error: Command failed with exit code 6: npm run cypress:run:chrome
    at makeError (/Users/lars/Documents/GitHub/cwa-website/node_modules/start-server-and-test/node_modules/execa/lib/error.js:60:11)
    at handlePromise (/Users/lars/Documents/GitHub/cwa-website/node_modules/start-server-and-test/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  shortMessage: 'Command failed with exit code 6: npm run cypress:run:chrome',
  command: 'npm run cypress:run:chrome',
  escapedCommand: '"npm run cypress:run:chrome"',
  exitCode: 6,
  signal: undefined,
  signalDescription: undefined,
  stdout: undefined,
  stderr: undefined,
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}
ERROR: "test:chrome:phase2" exited with 1.

[MikeMcC399]

@larswmh

Thank you also for testing with Mac M1!

Your results are different to @mtb77. The applink.cy.js test that failed for him worked for you. Conversely tests which failed in your environment (eventRegistration.cy.js and hybrid/check_links.cy.js) were successful for @mtb77.

Why do you think that your problem is due to the gulp-imagemin package? Does npm ls gulp-imagemin show [email protected] or do you have a different version installed? Dependabot PR https://github.com/corona-warn-app/cwa-website/pull/3108 is trying to update to gulp-imagemin from 7.1.0 to 8.0.0, but that fails. The Dependabot PR should however not have any influence if you ran npm ci in the cloned cypress-10 branch.

If you execute npm start in your environment does the web build and run without errors?

If the web is building correctly then I have the same question for you as for @mtb77:

What is the result of npm run test:open followed by individually running eventRegistration.cy.js and hybrid/check_links.cy.js?

[larswmh]

@MikeMcC399

thanks for your help! I'm actually new to using a Mac and usually work on Windows, but as I was surprised about the failed test and was curious why it would happen, I tested for myself and it seems like I went into a much bigger problem.

The version for gulp-imagemin is 7.1.0. If ran on 8.0.0 without any further changes, I'm getting the following error:

Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-imagemin/index.js from /Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-load-plugins/index.js not supported.
Instead change the require of /Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-imagemin/index.js in /Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-load-plugins/index.js to a dynamic import() which is available in all CommonJS modules.
    at requireFn (/Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-load-plugins/index.js:72:14)
    at Object.get [as imagemin] (/Users/lars/Documents/GitHub/cwa-website/node_modules/gulp-load-plugins/index.js:112:41)
    at images_minify (/Users/lars/Documents/GitHub/cwa-website/gulpfile.js:270:30)
    at bound (node:domain:421:15)
    at runBound (node:domain:432:12)
    at asyncRunner (/Users/lars/Documents/GitHub/cwa-website/node_modules/async-done/index.js:55:18)

All that comes below is on 7.1.0, as this seems to be the correct version and everyone else has most likely used the same. When running npm start on your branch (or master, doesn't make a difference in that case), I get the following error:

[12:29:42] Error in plugin "gulp-imagemin"
Message:
    spawn Unknown system error -86
Details:
    errno: -86
    code: Unknown system error -86
    syscall: spawn
    fileName: /Users/lars/Documents/GitHub/cwa-website/src/assets/img/faq/EU_digital_vaccination_certificate.jpg
    domainEmitter: [object Object]
    domainThrown: false

Deleting the entire src/assets/img directory lets it run through and start, although missing a lot of images.

Deleting just the .jpg and .jpeg leads into this error:

[12:37:08] Error in plugin "gulp-webp"
Message:
   PNG support not compiled. Please install the libpng development package before building.
Error! Could not process file /private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/e9cd7926-8858-400e-8d57-732a4a9bf4cc
Error! Cannot read input picture file '/private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/e9cd7926-8858-400e-8d57-732a4a9bf4cc'

Details:
   code: 255
   killed: false
   stdout: 
   stderr: PNG support not compiled. Please install the libpng development package before building.
Error! Could not process file /private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/e9cd7926-8858-400e-8d57-732a4a9bf4cc
Error! Cannot read input picture file '/private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/e9cd7926-8858-400e-8d57-732a4a9bf4cc'

   failed: true
   signal: null
   cmd: /Users/lars/Documents/GitHub/cwa-website/node_modules/cwebp-bin/vendor/cwebp -quiet -mt -o /private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/b31ac84c-e707-4036-b7bf-dcc18b8f1197 /private/var/folders/3h/rt53tpd11fz405c_wt3glpkw0000gn/T/e9cd7926-8858-400e-8d57-732a4a9bf4cc
   timedOut: false
   fileName: /Users/lars/Documents/GitHub/cwa-website/src/assets/img/app-mockup.png
   domainEmitter: [object Object]
   domainThrown: false

It seems that unlike previously assumed, .jpg and .jpeg are not the only problem.

Anyway, continuing the way that I previously made the test, the following errors occurred: check_links: As the directory src/assets/img was deleted in order to get the site to run locally, the 4 pages including assets from those directories are failing: de/screenshots, en/screenshots, de/faq/, en/faq. So this one is definitely related to that and would be sorted out with the gulp-imagemin problem being fixed.

eventRegistration: I'm assuming this is also related to the deleted images. The result PDF with the QR code on it is not complete as the assets that are supposed to be used for it are missing. Apart from that, there is no problem with the event registration. The button, highlighted in the error, behaves like it should.

---

In short, your PR is running fine and the problems that I encountered are unrelated to the changes you have made. As @mtb77 ran the two test without a problem, it seems like a problem on my end that I need to fix. It is unclear to me how his error occurred when I didn't get it, though. From my research so far, it doesn't look like much people encountered the same issue so far.

[MikeMcC399]

@larswmh

Thanks for providing the details on your tests! There is some mention of plugin errors on M1 in https://github.com/sindresorhus/gulp-imagemin/issues, so it looks like there may be a generic problem with the web on M1.

It does seem that this is not a Cypress problem, if the web will not build properly. Was @mtb77 perhaps using Rosetta emulation when he ran his tests?

If the web does not build on M1, then I'm not sure that we have any advantage in moving from Cypress 9 to Cypress 10, in which case perhaps we should drop this PR and stay on Cypress 9?

If it is important to be able to build the web on M1, then this should be a separate issue / enhancement request to be looked at as a pre-requisite to a Cypress migration to version 10.

[Ein-Tim]

I'm using a M1 MacBook Air and regularly run into different build errors when building the website. I also didn't get the Cypress test suite to work on my Mac and have given up trying to.

What I always do when I encounter such an error: Restart the whole machine. This solved problems for me I never thought a restart would solve.

Also, I found a correlation between freeing space on my disk with CleanMyMacX & build errors, so I assume some cleaning tools also remove necessary files which are then restored by installing them again using the command nmp prompts you to enter.

But this just as a side comment of my experience.

[MikeMcC399]

Converting back to Draft status since there are blocking issues outside of this PR:

1. The web does not build natively under Apple M1 ARM (see https://github.com/corona-warn-app/cwa-website/pull/3137#issuecomment-1278870513). If the web is not built correctly, then tests will fail.
2. ~~PR https://github.com/corona-warn-app/cwa-website/pull/3138 causes the Cypress test suite to fail under Cypress 9.7 so if the current master branch is merged into this PR, then Cypress 10.10 will also fail.~~ Fixed through PR https://github.com/corona-warn-app/cwa-website/pull/3147

[MikeMcC399]

Closing, due to deferral of the issue. See on-hold comment https://github.com/corona-warn-app/cwa-website/issues/3100#issuecomment-1282465529.

[MikeMcC399]

@Ein-Tim

I'm using a M1 MacBook Air and regularly run into different build errors when building the website. ...

What I always do when I encounter such an error: Restart the whole machine. This solved problems for me I never thought a restart would solve.

Also, I found a correlation between freeing space on my disk with CleanMyMacX & build errors, so I assume some cleaning tools also remove necessary files which are then restored by installing them again using the command nmp prompts you to enter.

Are you currently having problems to build the web? Could you open a new issue if you get a problem that npm run build fails in a way that you cannot resolve on your Apple M1 ARM environment?

[Ein-Tim]

@MikeMcC399 Currently it seems to work, but should I encounter an issue which I can't resolve, I will open an issue!

[MikeMcC399]

@Ein-Tim

Currently it seems to work, but should I encounter an issue which I can't resolve, I will open an issue!

Thank you! Perhaps there will an opportunity in future months to try the Cypress migration again? (Maybe when there is another new Cypress version released.) I would check beforehand that the web can be built in M1 though, otherwise there is no advantage.