cypress-cloud
cypress-cloud copied to clipboard
currents-dev
[CSR-791] cypress-cloud is not compatible with cypress 13.x
Before opening, please confirm:
- [X] I have searched for duplicate or closed issues and discussions.
- [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
- [X] I acknowledge that I will attach a full debug log, otherwise the issue will be closed with no response.
Environment information
# Put output below this line
System:
OS: Linux 6.1 Debian GNU/Linux 12 (bookworm) 12 (bookworm)
CPU: (8) x64 Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
Memory: 18.94 GB / 31.16 GB
Container: Yes
Shell: 5.2.15 - /bin/bash
Binaries:
Node: 18.16.0 - /usr/bin/node
Yarn: 1.22.19 - /usr/bin/yarn
npm: 9.5.1 - /usr/bin/npm
Browsers:
Chrome: 114.0.5735.90
Chromium: 116.0.5845.140
npmPackages:
eslint-config-custom: latest => 0.0.0
prettier: latest => 3.0.2
turbo: ^1.10.13 => 1.10.13
npmGlobalPackages:
azurite: 3.23.0
corepack: 0.17.0
gulp-cli: 2.3.0
npm: 9.5.1
samfirm: 0.2.0
Describe the bug
Due to this cypress commit that introduces some breaking API changes in cypress 13.x, I noticed two issues when using cypress-cloud with it:
- Executed cypress tests are always reported as failing whatever their real statuses are.
See sample cypress-cloud output put below:
$ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36
yarn run v1.22.19
$ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36
WARNING Failed to load config file, falling back to the default config. Attempted locations: [
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs'
]
Copyright (C) 2023 Currents Software Inc https://currents.dev
This is free software, and you are welcome to redistribute it under certain
conditions. This program comes with no warranty. Parts of this program are MIT
licensed. Refer to the license for details
https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
Discovered 1 spec files
Tags: false; Group: false; Parallel: true; Batch Size: 3
Connecting to cloud orchestration service...
🎥 Run URL: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7
====================================================================================================
Running: cypress/e2e/home.cy.js (1/1)
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.1.0 │
│ Browser: Electron 106 (headless) │
│ Node Version: v18.16.0 (/usr/bin/node) │
│ Specs: 1 found (home.cy.js) │
│ Searched: cypress/e2e/home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: home.cy.js (1 of 1)
Home Page Tests
✓ should have focus on search form after page load
✓ should display positive stats for each category
✓ should display null counters and hide history graphs when storage is empty
✓ should hide counters when data is missing
✓ should redirect to search page when submitting search form
5 passing (6s)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 5 │
│ Passing: 5 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 6 seconds │
│ Spec Ran: home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ home.cy.js 00:06 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:06 5 5 - - -
Reporting results and artifacts in background...
====================================================================================================
Cloud Run Finished
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Spec Tests Passing Failing Pending Skipped │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ cypress/e2e/home.cy.js 0ms 1 - 1 - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ 1 of 1 failed 0ms 1 - 1 - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
🏁 Recorded Run: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
After reading cypress-cloud code, it seems that issue can be fixed using the following diff:
diff --git a/packages/cypress-cloud/lib/results/results.ts b/packages/cypress-cloud/lib/results/results.ts
index f9380de..635f0b4 100644
--- a/packages/cypress-cloud/lib/results/results.ts
+++ b/packages/cypress-cloud/lib/results/results.ts
@@ -12,11 +12,6 @@ import { ConfigState } from "../state";
const debug = Debug("currents:results");
-export const isSuccessResult = (
- result: CypressResult
-): result is CypressCommandLine.CypressRunResult => {
- return result.status === "finished";
-};
export const getScreenshotsSummary = (
tests: CypressCommandLine.TestResult[] = []
@@ -282,9 +277,6 @@ export function getCypressRunResultForSpec(
spec: string,
cypressResult: CypressResult
): CypressCommandLine.CypressRunResult | undefined {
- if (!isSuccessResult(cypressResult)) {
- return;
- }
const run = cypressResult.runs.find((r) => r.spec.relative === spec);
if (!run) {
- Screenshots fail to be processed.
Once the fix above applied, there is an error when cypress-cloud attempts to process test screenshots, see output below:
$ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38
yarn run v1.22.19
$ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38
WARNING Failed to load config file, falling back to the default config. Attempted locations: [
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs'
]
Copyright (C) 2023 Currents Software Inc https://currents.dev
This is free software, and you are welcome to redistribute it under certain
conditions. This program comes with no warranty. Parts of this program are MIT
licensed. Refer to the license for details
https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
Discovered 1 spec files
Tags: false; Group: false; Parallel: true; Batch Size: 3
Connecting to cloud orchestration service...
🎥 Run URL: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae
====================================================================================================
Running: cypress/e2e/home.cy.js (1/1)
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.1.0 │
│ Browser: Electron 106 (headless) │
│ Node Version: v18.16.0 (/usr/bin/node) │
│ Specs: 1 found (home.cy.js) │
│ Searched: cypress/e2e/home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: home.cy.js (1 of 1)
Home Page Tests
✓ should have focus on search form after page load
✓ should display positive stats for each category
✓ should display null counters and hide history graphs when storage is empty
✓ should hide counters when data is missing
✓ should redirect to search page when submitting search form
5 passing (6s)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 5 │
│ Passing: 5 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 6 seconds │
│ Spec Ran: home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ home.cy.js 00:06 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:06 5 5 - - -
Reporting results and artifacts in background...
ERROR TypeError: Cannot read properties of undefined (reading 'flatMap')
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:26:21
at Array.flatMap (<anonymous>)
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:25:19
at Array.flatMap (<anonymous>)
at getScreenshotsSummary (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:24:16)
at getInstanceResultPayload (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:62:18)
at getReportResultsTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/uploadResults.ts:22:13)
at createReportTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/reportTask.ts:26:3)
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:55:7
at Array.forEach (<anonymous>)
at runTillDone (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:54:7)
====================================================================================================
Cloud Run Finished
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Spec Tests Passing Failing Pending Skipped │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ undefined 6.2s 5 5 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ All specs passed! 6.2s 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
🏁 Recorded Run: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae
Done in 13.82s.
The fix for this one is not straightforward to me.
Expected behavior
cypress 13.x should be supported.
Command and Setup
Just run any cypress-cloud session with cypress 13.x and the reported errors will show up.
Full log and debug output
Is this also related to the issue that viewing a run in Sorry Cypress is not possible and shows the following error:
Please check out the upgrade guide: https://currents.dev/readme/integration-with-cypress/cypress-cloud/migration-to-cypress-13
I just tested the upgrade to cypress-cloud beta and tests are no longer reported as failing, see https://jenkins.softwareheritage.org/blue/organizations/jenkins/DWAPPS%2Fgitlab-builds/detail/gitlab-builds/302/pipeline/89/.
There is still the error about screenshots that appears after each test suite though but if you do not use that cypress feature it is not critical for running the tests.
I too am having issues with the latest build 1.10.0-beta.2, but not the same as above.
file:///C:/dev/src/my-project/node_modules/cypress-cloud/index.mjs:2273
..._11.cloneDeep(specResults),
^
TypeError: _11.cloneDeep is not a function
Hacking in a bit of debugging, it seems that _11 contains default, which contains cloneDeep - which seems to me to be imported as a commonjs module rather than esm (which is how all my project is configured).
EDIT:
In v1.9.4, the imports in index.mjs are in the format import _11 from "lodash";, whereas in same file in v1.10.0-beta.2, the imports are now import * as _11 from "lodash";
For us, after updating everything to the latest version we got the following error, and the specs are not executed:
11:47:24 Copyright (C) 2023 Currents Software Inc https://currents.dev
11:47:24 This is free software, and you are welcome to redistribute it under certain
11:47:24 conditions. This program comes with no warranty. Parts of this program are MIT
11:47:24 licensed. Refer to the license for details
11:47:24 https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
11:47:24
11:47:26
11:47:26
11:47:26 (Attempt 1 of 2) An uncaught error was detected outside of a test
11:47:26 1) An uncaught error was detected outside of a test
11:47:26
11:47:26 0 passing (910ms)
11:47:26 1 failing
11:47:26
11:47:26 1) An uncaught error was detected outside of a test:
11:47:26 Error: The following error originated from your test code, not from Cypress.
11:47:26
11:47:26 > Dynamic require of "@neuralegion/cypress-har-generator/commands" is not supported
11:47:26
11:47:26 When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
11:47:26
11:47:26 Cypress could not associate this error to any specific test.
11:47:26
Versions:
- "cypress": "13.1.0",
- "cypress-cloud": "1.9.4",
- "cypress-debugger": "^1.0.9",
The tests are working with
"cypress": "13.1.0",
"cypress-cloud": "1.10.0-beta.2",
"cypress-debugger": "1.0.7",`
and you can see the logs on the command line that the tests passed ok:
12:45:25 (Run Starting)
12:45:25
12:45:25 ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:25 │ Cypress: 13.1.0 │
12:45:25 │ Browser: Electron 106 (headless) │
12:45:25 │ Node Version: v16.16.0 (/usr/bin/node) │
12:45:25 │ Specs: 1 found (admin_auth.spec.ts) │
12:45:25 │ Searched: cypress/test/genially/admin/admin_auth.spec.ts │
12:45:25 │ Experiments: experimentalRunAllSpecs=true,experimentalMemoryManagement=true,experimentalWe… │
12:45:25 └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:25
12:45:25
12:45:25 ────────────────────────────────────────────────────────────────────────────────────────────────────
12:45:25
12:45:25 Running: admin_auth.spec.ts (1 of 1)
12:45:32
12:45:32
12:45:32 Admin/Auth/AdminUser
12:45:37 ✓ should login as authorized master user (5497ms)
12:45:40 ✓ should logout as authorized master user (2265ms)
12:45:40
12:45:40
12:45:40 2 passing (9s)
12:45:40
12:45:40
12:45:40 (Results)
12:45:40
12:45:40 ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40 │ Tests: 2 │
12:45:40 │ Passing: 2 │
12:45:40 │ Failing: 0 │
12:45:40 │ Pending: 0 │
12:45:40 │ Skipped: 0 │
12:45:40 │ Screenshots: 0 │
12:45:40 │ Video: false │
12:45:40 │ Duration: 8 seconds │
12:45:40 │ Spec Ran: admin_auth.spec.ts │
12:45:40 └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40
12:45:40
12:45:40 resize: can't open terminal /dev/tty
12:45:40 ====================================================================================================
12:45:40
12:45:40 (Run Finished)
12:45:40
12:45:40
12:45:40 Spec Tests Passing Failing Pending Skipped
12:45:40 ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40 │ ✔ admin_auth.spec.ts 00:08 2 2 - - - │
12:45:40 └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40 ✔ All specs passed! 00:08 2 2 - - -
12:45:40
12:45:40
12:45:40 Reporting results and artifacts in background...
12:45:40
12:45:40 ERROR TypeError: Cannot read properties of undefined (reading 'flatMap')
12:45:40 at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:27:21
12:45:40 at Array.flatMap (<anonymous>)
12:45:40 at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:26:19
12:45:40 at Array.flatMap (<anonymous>)
12:45:40 at getRunScreenshots (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:25:16)
12:45:40 at getInstanceResultPayload (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/results.ts:66:18)
12:45:40 at getReportResultsTask (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/results/uploadResults.ts:26:27)
12:45:40 at createReportTask (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/reportTask.ts:30:5)
12:45:40 at /home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/runner.ts:58:7
12:45:40 at Array.forEach (<anonymous>)
12:45:40 at runTillDone (/home/ubuntu/workspace/workspace/ressED_qa_PD-13026_cy13_recorder/mono-genially/qa/e2e/node_modules/lib/runner/runner.ts:57:14)
12:45:40
12:45:40
12:45:40 ====================================================================================================
12:45:40
12:45:40
12:45:40 Cloud Run Finished
12:45:40
12:45:40 ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
12:45:40 │ Spec Tests Passing Failing Pending Skipped │
12:45:40 ├────────────────────────────────────────────────────────────────────────────────────────────────┤
12:45:40 │ ✔ cypress/test/genially/admin/ad 8.6s 2 2 - - - │
12:45:40 │ min_auth.spec.ts │
12:45:40 ├────────────────────────────────────────────────────────────────────────────────────────────────┤
12:45:40 │ ✔ All specs passed! 8.6s 2 2 - - - │
12:45:40 └────────────────────────────────────────────────────────────────────────────────────────────────┘
12:45:40
12:45:40 🏁 Recorded Run: https://dashboard-sorrycypress.genially.com/run/4acaa09f2aa7f36f26d5acbec86c675e
The problem is, that the results do not appears on sorry cypress (or the mongodb) and you get 0 tests executed (and the specs runs indefinitely)
@estefafdez thanks for reporting - will keep cypress-debugger issue separated
@Roemer, @anlambert and @richardmward please check out 1.10.0-beta.3 also, please submit the debug logs when you face issues - would be much easier for us, and thanks for helping with polishing the integration!
@agoldis the debugger issue is already reported there https://github.com/currents-dev/cypress-debugger/issues/59 The problem is that you can run the tests in Cy13 but you can't see the results in sorry cypress, Something seems to be missing as well.
Anyone else experiencing tests just getting stuck and never finishing? I'm using [email protected] with [email protected], having followed the migration guide, and the tests start, but it just hangs after the first one. This happens both in open and run mode, and if I remove the cypress-cloud/support and cypress-cloud/plugin parts, the tests work. Any idea what is going on?
@agoldis I just upgraded to beta 3 and executed the cypress tests of the project I am working on, all good from my side and it indeed fixes the screenshots processing bug, see https://jenkins.softwareheritage.org/blue/organizations/jenkins/DWAPPS%2Fgitlab-builds/detail/gitlab-builds/306/pipeline/88.
Nevertheless, I only use sorry-cypress as tests orchestrator and do not use its reporting dashboards so I might miss other issues.
@anlambert what did you see on the cypress dashboard when the run is finished? did you see the results there?
@estefafdez @richardmward Please check out 1.10.0-beta.4 - it has a few additional fixes.
Also, the corrupted runs could have incompatible data, they should be removed from the DB.
@andreiho we need more data to explore the case - the debug logs would help
@estefafdez regarding the sorry-cypress dashboard feature, I bumped cypress-cloud to 1.10.0-beta.4 and tested it locally using that docker-compose.yml file to run the sorry-cypress kit. Everything seems to work as expected, see screenshots below:
Anyone else experiencing tests just getting stuck and never finishing? I'm using
[email protected]with[email protected], having followed the migration guide, and the tests start, but it just hangs after the first one. This happens both inopenandrunmode, and if I remove thecypress-cloud/supportandcypress-cloud/pluginparts, the tests work. Any idea what is going on?
@andreiho I also experience this with npx cypress open The test hangs whenever I put import "cypress-cloud/support" in my e2e.js file but if I remove it there will be no problem with cypress runner.
One thing I also noticed is that whenever I check the cypress-cloud/support its index.d.ts file contains an empty export
here's additional info: cypress version: 13.1.0 cypress-cloud version: 1.10.0-beta.4 node version: 20.5.0
@bilarallen thanks for elaborating on the issue. If that is reproducible, please share a standalone example or and debug logs (https://currents.dev/readme/integration-with-cypress/troubleshooting)
@bilarallen thanks for elaborating on the issue. If that is reproducible, please share a standalone example or and debug logs (https://currents.dev/readme/integration-with-cypress/troubleshooting)
Hi @agoldis This can only be replicated when running a test via cypress runner through npx cypress open command, here's my envinfo and I'll also include steps to replicate since this only be replicated when using cypress runner
System:
OS: Linux 5.10 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
CPU: (16) x64 AMD Ryzen 7 5800U with Radeon Graphics
Memory: 4.06 GB / 6.71 GB
Container: Yes
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 20.5.0 - ~/.nvm/versions/node/v20.5.0/bin/node
npm: 9.8.0 - ~/.nvm/versions/node/v20.5.0/bin/npm
Browsers:
Chrome: 111.0.5563.64
npmPackages:
@babel/eslint-parser: ^7.21.8 => 7.21.8
@currents/cli: ^4.0.3 => 4.0.3
@cypress/angular: 0.0.0-development
@cypress/mount-utils: 0.0.0-development
@cypress/react: 0.0.0-development
@cypress/react18: 0.0.0-development
@cypress/svelte: 0.0.0-development
@cypress/vue: 0.0.0-development
@cypress/vue2: 0.0.0-development
@cypress/webpack-batteries-included-preprocessor: ^3.0.2 => 3.0.2
@cypress/webpack-preprocessor: ^5.17.0 => 5.17.0
@faker-js/faker: ^7.6.0 => 7.6.0
babel-eslint: ^10.1.0 => 10.1.0
cy-verify-downloads: ^0.1.16 => 0.1.16
cypress: ^13.1.0 => 13.1.0
cypress-cloud: ^1.10.0-beta.4 => 1.10.0-beta.4
cypress-delete-downloads-folder: 0.0.3 => 0.0.3
cypress-network-idle: ^1.14.2 => 1.14.2
cypress-real-events: ^1.10.1 => 1.10.1
cypress-recurse: ^1.27.0 => 1.27.0
eslint: ^8.39.0 => 8.39.0
moment: ^2.29.4 => 2.29.4
moment-timezone: ^0.5.43 => 0.5.43
numeral: ^2.0.6 => 2.0.6
swagger-client: ^3.18.5 => 3.18.5
uuid: ^9.0.0 => 9.0.0 (8.3.2)
xvfb: ^0.4.0 => 0.4.0
npmGlobalPackages:
corepack: 0.19.0
npm: 9.8.0
Steps to replicate when using cypress runner and importing cypress-cloud/support in e2e.js file:
1. Install and use cypress version 12.3.1
2. Install and use cypress-cloud version 1.10.0-beta.4
3. Go to cypress > support > e2e.js file
4. import `cypress-cloud/support` in e2e.js file
5. Run a test using cypress runner via `npx cypress open` command and select electron browser
6. Wait for test execution to finish and observe if it hang
7. Remove import `cypress-cloud/support` in e2e.js file and repeat steps 5 to 6
@agoldis @bilarallen : I'm experiencing the same issue with cypress open. Is there any solution yet?
Ok, for me is finally working with:
"cypress": "13.1.0",
"cypress-cloud": "1.10.0-beta.4",
"cypress-debugger": "1.0.7",
@agoldis thanks!
@agoldis @bilarallen I've been trying to create a reproducible example, but can't manage to. It appears to be related to some of the custom behaviour we have implemented to deal with some technicalities in our application. But I can't quite explain how importing the cypress-cloud code results in the tests hanging and cypress freezing completely. I'll try to give a bit more context into our situation.
We have a few custom commands which we run on most tests, in order to perform certain actions before proceeding with the test. In this case, the navigateTo command will wait for the page to finish loading, then wait for an element #alerts-holder to be present and attempt to clear any notifications using the clickNotificationButton command in order to prevent notifications covering up content which might need to be clicked in the test.
Now I've tried changing different things in these commands in an attempt to maybe fix some issues, but I can't quite get to the bottom of it, particularly why importing cypress-cloud ends up manifesting this way. It does however work if I simply just never call the clickNotificationButton command, even with cypress-cloud imported. The way the issue manifests is that the tests just freeze, along with the cypress browser, requiring a force quit of the process. It freezes so badly, that it makes the CPU on my M1 Pro go crazy.
This could of course be sub-optimal code on our part, but why would it only manifest itself this way while cypress-cloud is imported? I've tried looking at your plugin and support code, and with my limited knowledge I can't quite tell what could be causing issues there.
Our custom commands
export const navigateTo = (
routePath = '/',
options: { isLoggedIn?: boolean; force?: boolean } = {},
): void => {
const url = new URL(`${Cypress.env('BASE_URL')}${routePath}`);
const { force = false, isLoggedIn = true } = options;
cy.url().then((href) => {
if (force || href !== url.href) {
cy.visit(url.href);
cy.get('body').should('not.have.class', 'loading');
if (isLoggedIn) {
cy.get('#alerts-holder', { timeout: 20000 }).should('exist');
cy.clickNotificationButton();
}
}
});
};
export const clickNotificationButton = (
buttonText?: string,
): Cypress.Chainable<JQuery<HTMLElement>> => {
return cy
.get('#alerts-holder')
.as('alerts')
.then(($aw) => {
if (buttonText) {
cy.get('@alerts')
.contains(button, buttonText)
.as('button')
.then(() => {
cy.get('@button').click();
});
} else if ($aw.find('[aria-label="close"]').length) {
cy.safeClick(
() => cy.get('#alerts-holder button[aria-label="close"]'),
{
multiple: true,
},
);
}
});
};
export const safeClick = (
cySelectorFn: () => Cypress.Chainable<JQuery<HTMLElement>>,
clickOpts?: Partial<Cypress.ClickOptions>,
): Cypress.Chainable<JQuery<HTMLElement>> => {
return cy
.waitUntil(
() => {
return cySelectorFn()
.as('clickable')
.wait(10) // https://github.com/cypress-io/cypress/issues/7306
.then(($el) => Cypress.dom.isAttached($el));
},
{ timeout: 1000, interval: 10 },
)
.get('@clickable')
.click(clickOpts);
};
Okay after poking around a bit more, I think I may have found the culprit. You can probably disregard most of what I said in my previous message, although I still think there's some relation between the how tests are written, and what happens further down.
The cypress-cloud/support file which you can see here has this piece of code attempting to stringify the entire test object using fast-safe-stringify. Without this safeStringify call, the tests run perfectly. I haven't looked deeper into what exactly happens, but the full test object looks like a fairly complex thing with circular references, which the lib might struggle with (even though it's designed for that), possibly resulting in a recursive loop of sorts and blocking everything else from executing.
cy.task(
`currents:test:after:run`,
safeStringify({
...test,
fullTitle: test.fullTitle()
}),
{
log: false
}
);
Screenshot of the `test` object in my case
@andreiho thanks a lot for the detailed response. Any chance you can share the test / sample with me somehow? Another approach would be to put a breakpoint right before stringifying the result and make sure that the function returns.
@agoldis I did try a breakpoint and calling the stringify function in the console, and it never returns, it just hangs indefinitely and the browser becomes virtually unusable, requiring force quit.
Haven't been able to put something useful together to share, as it seems somewhat related to some of the more complex orchestration of our application, with large number of requests made on load and various mocks and intercepts. The test itself is as simple as it gets, I tried running it on a dummy HTML webpage and everything works fine, which makes me think it's something else in the background that is part of the test object which the stringifier doesn't like. I might try manually inspecting that test object before/after commenting out some of the code I previously mentioned which seems to unblock the tests, and see what the differences are.
@andreiho would you be able to join a quick support call?
@agoldis I'm not able right now, it's late here in Europe, but I'll try to do a bit more digging tomorrow at work and maybe see if I can somehow give you access to something reproducible.
@agoldis I went through the test object and tried running safeStringify on each property, to see where it chokes. The 2 things that fail are commands and parent (which circularly references the same test object).
A few examples of things that fail. In these cases I specifically went through all the properties and the stringifier only freezes on these exact properties, there are probably more scenarios however. To me it just seems that the unpredictability and complexity of this test object is maybe too much for the stringifier library, and maybe another solution should be considered.
1. Commands that have the `$el` property referencing a DOM element
2. Intercept commands that use a regex matcher (`hostname` in this case)
Anyhow, I created a super simple repro example, where I load the website of a tool we use internally, which happens to make tons of requests on load. The test itself is nothing but just loading the page and asserting a button is there. Run this test with cypress open and it should freeze (for some reason it works with run, although in my application it doesn't). If you add a breakpoint in the support/index.js file before stringifying and you try to manually stringify the test.commands array, it should freeze and never return. More specifically the last command in this case asserting the button is visible is the one that fails: 55: {subject: jQuery.fn.init(1), $el: jQuery.fn.init(1), _error: undefined, name: 'assert', message: 'expected **<a.button.>** to be **visible**', …}