cypress
cypress copied to clipboard
Published
20 hours ago •
cypress-io
cypress-io
CT Performance Improvements: v10 slower than v9
Edit from @lmiller1990: this is getting worked on: https://github.com/cypress-io/cypress/pull/23104. Component Testing performance will be improved significantly, initially released under an experimental flag to let use continue making incrementally performance updates.
Current behavior
After upgrading to the latest version, we have an issue with slowness when we run cypress run --component. The tests themselves are not slow, but it seems there is some delay between specs runs as you can see in this video 👇 .
https://user-images.githubusercontent.com/6830426/174029644-aa31da35-ee35-4a04-8474-b0eab9a72869.mp4
We were not able to detect the exact issue, but we would appreciate any help 🙏
I was trying to run the tests with DEBUG and this might be potentially the issue 👇
Desired behavior
The tests should run smoothly without any delay.
Test code to reproduce
We use cypress on public repo https://github.com/toptal/picasso.
We have cypress 10 on the current master.
yarn
yarn test:integration
for debugging it is faster to run
# these need to be run only once
yarn
yarn build:package
yarn test:setup cypress run --component
Cypress Version
10.1.0
Other
We use cypress with Happo for visual testing. The first idea was that it might be the problem since Happo didn't adapt to v10 yet, but it is not the case, I have tried to remove it from the repo and it didn't help.
We're seeing a 3x increase in more complex spec files (from 1651ms to 4887ms) after upgrading to v10 (10.1.0, vue and vite)
To make sure, that problem is not on our side, I have cloned your example from https://github.com/cypress-io/cypress-component-testing-apps/tree/main/react-webpack5-js and I think it has the same problem.
- webpack compiled in 1650ms
- All specs passed in 4sec
But the whole process of npx cypress run --component --browser=electron took 14sec
I've not fully investigated, but our tests suite went from taking around 14 minutes up to 19 minutes since upgrading to 10.1.0 from 9.7.0
@TomasSlama thank you for the report, and thanks all for the corroborating comments. I have the picasso tests up and running and will report back my findings.
There is definitely a slowdown here. Running your picasso tests, I see that:
- CT tests on commit f692ca02 (with Cypress 10.1.0) take about ~200s
- Command used:
yarn test:setup cypress run --component
- Command used:
- CT tests on commit b90712743a (with Cypress 9.7.0) take about ~80s
- Command used:
yarn test:setup cypress run-ct
- Command used:
It appears that with 10.x we do some cleanup work/reset some browser state between spec executions that is the bulk of the added time. This time increases linearly with the number of specs included with a given run. The debug log output from the 10.1.0 run is provided below.
Thanks again for the report. I'm going to direct this issue to the CT team to investigate mitigating factors.
10.1.0 debug output between spec run start and actually executing tests in the spec (click me)
Running: Autocomplete.spec.tsx (3 of 60)
[HAPPO] Happo is disabled. Here's how to enable it:
- Use the `happo-e2e` wrapper.
- Set `HAPPO_ENABLED=true`.
Docs:
https://docs.happo.io/docs/cypress#usage-with-cypress-run
https://docs.happo.io/docs/cypress#usage-with-cypress-open
cypress:server:run about to run spec { spec: { fileExtension: '.tsx', baseName: 'Autocomplete.spec.tsx', fileName: 'Autocomplete', specFileExtension: '.spec.tsx', relativeToCommonRoot: 'Autocomplete.spec.tsx', specType: 'component', name: 'cypress/component/Autocomplete.spec.tsx', relative: 'cypress/component/Autocomplete.spec.tsx', absolute: '/Users/tyler/cypress-support/picasso/cypress/component/Autocomplete.spec.tsx' }, isHeadless: true, browser: { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.', isHeadless: true, isHeaded: false } } +2ms
cypress:server:run video recording has been disabled. video: false +2ms
cypress:server:run waiting for socket to connect and browser to launch... +0ms
cypress:server:run waiting for socket connection... { id: 'rt3xk1lond' } +0ms
cypress:server:run browser launched +0ms
cypress:server:open_project resetting project state, preparing to launch browser electron for spec { fileExtension: '.tsx', baseName: 'Autocomplete.spec.tsx', fileName: 'Autocomplete', specFileExtension: '.spec.tsx', relativeToCommonRoot: 'Autocomplete.spec.tsx', specType: 'component', name: 'cypress/component/Autocomplete.spec.tsx', relative: 'cypress/component/Autocomplete.spec.tsx', absolute: '/Users/tyler/cypress-support/picasso/cypress/component/Autocomplete.spec.tsx' } options { width: 1280, height: 720, show: false, onCrashed: [Function: onCrashed], onNewWindow: [Function: onNewWindow], automationMiddleware: { onBeforeRequest: [Function: onBeforeRequest], onAfterResponse: [Function: onAfterResponse] }, projectRoot: '/Users/tyler/cypress-support/picasso', shouldLaunchNewTab: true, onWarning: [Function (anonymous)] } +3s
cypress:server:project resetting project instance /Users/tyler/cypress-support/picasso +63ms
cypress:network:cors Parsed URL { port: '80', tld: '', domain: '' } +6ms
cypress:network:cors Parsed URL { port: '52701', tld: 'localhost', domain: '' } +0ms
cypress:server:remote-states setting remote state { auth: undefined, origin: 'http://localhost:52701', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for http://localhost:52701 +6ms
cypress:network:cors Parsed URL { port: '52701', tld: 'localhost', domain: '' } +1ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:52701', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:52701 +1ms
cypress:server:project_utils returning spec url http://localhost:8080/__/#/specs/runner?file=cypress/component/Autocomplete.spec.tsx +3s
cypress:server:open_project open project url http://localhost:8080/__/#/specs/runner?file=cypress/component/Autocomplete.spec.tsx +1ms
cypress:server:project project has config { configFile: '/Users/tyler/cypress-support/picasso/cypress.config.ts', env: { codeCoverageTasksRegistered: true }, testingType: 'component', defaultCommandTimeout: 6000, fixturesFolder: '/Users/tyler/cypress-support/picasso/cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: '/Users/tyler/cypress-support/picasso/cypress/screenshots', video: false, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: '/Users/tyler/cypress-support/picasso/cypress/support/component.ts', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: { module: [Object], resolve: [Object] } }, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso', rawJson: { defaultCommandTimeout: 6000, fixturesFolder: './cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: './cypress/screenshots', video: false, component: { viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: [Object] }, e2e: { setupNodeEvents: '[Function setupNodeEvents]', baseUrl: 'http://0.0.0.0:3000', slowTestThreshold: 10000, specPattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', supportFile: 'cypress/support/e2e.{js,ts}' }, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: [Object] }, envFile: {}, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso' }, morgan: false, isTextTerminal: true, socketId: 'rt3xk1lond', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, clientCertificates: [], downloadsFolder: '/Users/tyler/cypress-support/picasso/cypress/downloads', execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/Users/tyler/cypress-support/picasso', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'darwin', pageLoadTimeout: 60000, port: 52701, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/Users/tyler/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, scrollBehavior: 'top', supportFolder: '/Users/tyler/cypress-support/picasso/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, videoCompression: 32, videosFolder: '/Users/tyler/cypress-support/picasso/cypress/videos', videoUploadOnPasses: true, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '102.0.5005.115', path: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', minSupportedVersion: 64, majorVersion: 102 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '100.0.4896.0', path: '/Applications/Chromium.app/Contents/MacOS/Chromium', minSupportedVersion: 64, majorVersion: 100 }, { name: 'chrome', family: 'chromium', channel: 'beta', displayName: 'Chrome Beta', version: '103.0.5060.53', path: '/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: '/Applications/Firefox.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 101, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'dev', displayName: 'Firefox Developer Edition', version: '100.0', path: '/Applications/Firefox Developer Edition.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'nightly', displayName: 'Firefox Nightly', version: '100.0a1', path: '/Applications/Firefox Nightly.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/Users/tyler/Library/Caches/Cypress/10.1.0/Cypress.app/Contents/Resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.1.0', xhrRoute: '/xhrs/', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: false, from: 'config' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 6000, from: 'config' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: { codeCoverageTasksRegistered: [Object] }, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: './cypress/fixtures', from: 'config' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'darwin', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: './cypress/screenshots', from: 'config' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,ts}', from: 'config' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: false, from: 'config' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 1024, from: 'config' }, viewportWidth: { value: 1280, from: 'config' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, specPattern: { value: 'cypress/component/*.spec.tsx', from: 'config' }, configFile: { value: '/Users/tyler/cypress-support/picasso/cypress.config.ts', from: 'plugin' }, testingType: { value: 'component' } }, componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:52701', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:52701', state: {} } +1ms
cypress:network:cors Parsed URL { port: '52701', tld: 'localhost', domain: '' } +1ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:52701', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:52701 +1ms
cypress:server:open_project launching browser: { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.', isHeadless: true, isHeaded: false }, spec: cypress/component/Autocomplete.spec.tsx +2ms
cypress:server:browsers getBrowserLauncher { browser: { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.', isHeadless: true, isHeaded: false } } +3s
cypress:server:browsers:electron open { browser: { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.', isHeadless: true, isHeaded: false }, url: 'http://localhost:8080/__/#/specs/runner?file=cypress/component/Autocomplete.spec.tsx' } +29ms
cypress:server:saved_state noop saved state +3s
cypress:server:browsers:electron received saved state {} +0ms
cypress:server:browsers:electron browser window options { width: 1280, height: 720, show: false, automationMiddleware: { onBeforeRequest: [Function: onBeforeRequest], onAfterResponse: [Function: onAfterResponse] }, projectRoot: '/Users/tyler/cypress-support/picasso', shouldLaunchNewTab: true, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '102.0.5005.115', path: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', minSupportedVersion: 64, majorVersion: 102 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '100.0.4896.0', path: '/Applications/Chromium.app/Contents/MacOS/Chromium', minSupportedVersion: 64, majorVersion: 100 }, { name: 'chrome', family: 'chromium', channel: 'beta', displayName: 'Chrome Beta', version: '103.0.5060.53', path: '/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: '/Applications/Firefox.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 101, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'dev', displayName: 'Firefox Developer Edition', version: '100.0', path: '/Applications/Firefox Developer Edition.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'nightly', displayName: 'Firefox Nightly', version: '100.0a1', path: '/Applications/Firefox Nightly.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], userAgent: null, proxyUrl: 'http://localhost:52701', proxyServer: 'http://localhost:52701', socketIoRoute: '/__socket', chromeWebSecurity: false, isTextTerminal: true, downloadsFolder: '/Users/tyler/cypress-support/picasso/cypress/downloads', experimentalSessionAndOrigin: false, browser: { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.', isHeadless: true, isHeaded: false }, url: 'http://localhost:8080/__/#/specs/runner?file=cypress/component/Autocomplete.spec.tsx', resizable: false, x: null, y: null, devTools: false, minWidth: 100, minHeight: 100, contextMenu: true, partition: 'persist:run-85316', trackState: { width: 'browserWidth', height: 'browserHeight', x: 'browserX', y: 'browserY', devTools: 'isBrowserDevToolsOpen' }, webPreferences: { sandbox: true, partition: null, webSecurity: true, nodeIntegration: false, backgroundThrottling: false }, frame: true, recordFrameRate: null } +1ms
cypress:server:browsers:electron launching browser window to url: http://localhost:8080/__/#/specs/runner?file=cypress/component/Autocomplete.spec.tsx +1ms
cypress:server:browsers:electron debugger attached +9ms
cypress:server:browsers:electron clearing cache +0ms
cypress:server:socket-base socket-disconnecting transport close +108ms
cypress:server:socket-base socket-disconnect transport close +0ms
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 52759 } +1s
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 52770 } +0ms
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 52776 } +1ms
cypress:server:browsers:electron maybe recording video { onScreencastFrame: undefined } +121ms
cypress:server:browsers:electron debugger: enable Console and Network +1ms
cypress:server:project project has config { configFile: '/Users/tyler/cypress-support/picasso/cypress.config.ts', env: { codeCoverageTasksRegistered: true }, testingType: 'component', defaultCommandTimeout: 6000, fixturesFolder: '/Users/tyler/cypress-support/picasso/cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: '/Users/tyler/cypress-support/picasso/cypress/screenshots', video: false, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: '/Users/tyler/cypress-support/picasso/cypress/support/component.ts', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: { module: [Object], resolve: [Object] } }, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso', rawJson: { defaultCommandTimeout: 6000, fixturesFolder: './cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: './cypress/screenshots', video: false, component: { viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: [Object] }, e2e: { setupNodeEvents: '[Function setupNodeEvents]', baseUrl: 'http://0.0.0.0:3000', slowTestThreshold: 10000, specPattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', supportFile: 'cypress/support/e2e.{js,ts}' }, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: [Object] }, envFile: {}, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso' }, morgan: false, isTextTerminal: true, socketId: 'rt3xk1lond', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, clientCertificates: [], downloadsFolder: '/Users/tyler/cypress-support/picasso/cypress/downloads', execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/Users/tyler/cypress-support/picasso', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'darwin', pageLoadTimeout: 60000, port: 52701, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/Users/tyler/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, scrollBehavior: 'top', supportFolder: '/Users/tyler/cypress-support/picasso/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, videoCompression: 32, videosFolder: '/Users/tyler/cypress-support/picasso/cypress/videos', videoUploadOnPasses: true, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '102.0.5005.115', path: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', minSupportedVersion: 64, majorVersion: 102 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '100.0.4896.0', path: '/Applications/Chromium.app/Contents/MacOS/Chromium', minSupportedVersion: 64, majorVersion: 100 }, { name: 'chrome', family: 'chromium', channel: 'beta', displayName: 'Chrome Beta', version: '103.0.5060.53', path: '/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: '/Applications/Firefox.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 101, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'dev', displayName: 'Firefox Developer Edition', version: '100.0', path: '/Applications/Firefox Developer Edition.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'nightly', displayName: 'Firefox Nightly', version: '100.0a1', path: '/Applications/Firefox Nightly.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/Users/tyler/Library/Caches/Cypress/10.1.0/Cypress.app/Contents/Resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.1.0', xhrRoute: '/xhrs/', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: false, from: 'config' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 6000, from: 'config' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: { codeCoverageTasksRegistered: [Object] }, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: './cypress/fixtures', from: 'config' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'darwin', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: './cypress/screenshots', from: 'config' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,ts}', from: 'config' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: false, from: 'config' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 1024, from: 'config' }, viewportWidth: { value: 1280, from: 'config' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, specPattern: { value: 'cypress/component/*.spec.tsx', from: 'config' }, configFile: { value: '/Users/tyler/cypress-support/picasso/cypress.config.ts', from: 'plugin' }, testingType: { value: 'component' } }, componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:52701', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:52701', state: {} } +139ms
cypress:network:cors Parsed URL { port: '52701', tld: 'localhost', domain: '' } +139ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:52701', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:52701 +139ms
cypress:server:routes proxying static assets /__/assets/index.a96bfe29.js, params[0] index.a96bfe29.js +2s
cypress:server:routes proxying static assets /__/assets/index.0128a075.css, params[0] index.0128a075.css +1ms
cypress:server:server-base Got CONNECT request from localhost:8080 +2s
cypress:network:connect successfully connected { opts: { port: 52701, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +2s
cypress:server:util:socket_allowed allowing socket { localPort: 52799 } +380ms
cypress:server:server-base Got UPGRADE request from /__socket/?EIO=4&transport=websocket +3ms
cypress:server:util:socket_allowed is incoming request allowed? { isAllowed: true, reqUrl: '/__socket/?EIO=4&transport=websocket', remotePort: 52799, remoteAddress: '127.0.0.1' } +1ms
cypress:server:socket-base socket connected +407ms
cypress:server:socket-ct do onSocketConnection +3s
cypress:server:routes proxying static assets /__/assets/Specs.ec514639.js, params[0] Specs.ec514639.js +271ms
cypress:server:routes proxying static assets /__/assets/Runner.12af604b.js, params[0] Runner.12af604b.js +2ms
cypress:server:routes proxying static assets /__/assets/Runner.7591f1d0.css, params[0] Runner.7591f1d0.css +2ms
cypress:server:routes proxying static assets /__/assets/add-large_x16.34ef0540.js, params[0] add-large_x16.34ef0540.js +1ms
cypress:server:routes proxying static assets /__/assets/CreateSpecModal.7b82dc44.js, params[0] CreateSpecModal.7b82dc44.js +0ms
cypress:server:routes proxying static assets /__/assets/SpecPatterns.5862fb95.js, params[0] SpecPatterns.5862fb95.js +0ms
cypress:server:routes proxying static assets /__/assets/Switch.65e7a51e.js, params[0] Switch.65e7a51e.js +3ms
cypress:server:routes proxying static assets /__/assets/refresh_x16.c36d090a.js, params[0] refresh_x16.c36d090a.js +1ms
cypress:server:server-base Got CONNECT request from localhost:8080 +80ms
cypress:network:connect successfully connected { opts: { port: 52701, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +82ms
cypress:server:util:socket_allowed allowing socket { localPort: 52810 } +81ms
cypress:server:server-base Got UPGRADE request from /__socket-graphql +2ms
cypress:server:util:socket_allowed is incoming request allowed? { isAllowed: true, reqUrl: '/__socket-graphql', remotePort: 52810, remoteAddress: '127.0.0.1' } +1ms
cypress:server:project project has config { configFile: '/Users/tyler/cypress-support/picasso/cypress.config.ts', env: { codeCoverageTasksRegistered: true }, testingType: 'component', defaultCommandTimeout: 6000, fixturesFolder: '/Users/tyler/cypress-support/picasso/cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: '/Users/tyler/cypress-support/picasso/cypress/screenshots', video: false, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: '/Users/tyler/cypress-support/picasso/cypress/support/component.ts', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: { module: [Object], resolve: [Object] } }, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso', rawJson: { defaultCommandTimeout: 6000, fixturesFolder: './cypress/fixtures', chromeWebSecurity: false, screenshotsFolder: './cypress/screenshots', video: false, component: { viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: [Object] }, e2e: { setupNodeEvents: '[Function setupNodeEvents]', baseUrl: 'http://0.0.0.0:3000', slowTestThreshold: 10000, specPattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', supportFile: 'cypress/support/e2e.{js,ts}' }, viewportWidth: 1280, viewportHeight: 1024, slowTestThreshold: 250, specPattern: 'cypress/component/*.spec.tsx', supportFile: 'cypress/support/component.{js,ts}', setupNodeEvents: '[Function setupNodeEvents]', devServer: { framework: 'react', bundler: 'webpack', webpackConfig: [Object] }, envFile: {}, projectRoot: '/Users/tyler/cypress-support/picasso', projectName: 'picasso' }, morgan: false, isTextTerminal: true, socketId: 'rt3xk1lond', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, clientCertificates: [], downloadsFolder: '/Users/tyler/cypress-support/picasso/cypress/downloads', execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/Users/tyler/cypress-support/picasso', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'darwin', pageLoadTimeout: 60000, port: 52701, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/Users/tyler/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, scrollBehavior: 'top', supportFolder: '/Users/tyler/cypress-support/picasso/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, videoCompression: 32, videosFolder: '/Users/tyler/cypress-support/picasso/cypress/videos', videoUploadOnPasses: true, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/integration/**/*.spec.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '102.0.5005.115', path: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', minSupportedVersion: 64, majorVersion: 102 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '100.0.4896.0', path: '/Applications/Chromium.app/Contents/MacOS/Chromium', minSupportedVersion: 64, majorVersion: 100 }, { name: 'chrome', family: 'chromium', channel: 'beta', displayName: 'Chrome Beta', version: '103.0.5060.53', path: '/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: '/Applications/Firefox.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 101, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'dev', displayName: 'Firefox Developer Edition', version: '100.0', path: '/Applications/Firefox Developer Edition.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'firefox', family: 'firefox', channel: 'nightly', displayName: 'Firefox Nightly', version: '100.0a1', path: '/Applications/Firefox Nightly.app/Contents/MacOS/firefox', minSupportedVersion: 86, majorVersion: 100, warning: 'Your project has set the configuration option: \x1B[33mchromeWebSecurity\x1B[39m to \x1B[94mfalse\x1B[39m\n' + '\n' + 'This option will not have an effect in Firefox. Tests that rely on web security being disabled will not run as expected.' }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.75', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/Users/tyler/Library/Caches/Cypress/10.1.0/Cypress.app/Contents/Resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.1.0', xhrRoute: '/xhrs/', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: false, from: 'config' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 6000, from: 'config' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: { codeCoverageTasksRegistered: [Object] }, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: './cypress/fixtures', from: 'config' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'darwin', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: './cypress/screenshots', from: 'config' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,ts}', from: 'config' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: false, from: 'config' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 1024, from: 'config' }, viewportWidth: { value: 1280, from: 'config' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, specPattern: { value: 'cypress/component/*.spec.tsx', from: 'config' }, configFile: { value: '/Users/tyler/cypress-support/picasso/cypress.config.ts', from: 'plugin' }, testingType: { value: 'component' } }, componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:52701', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:52701', state: {} } +382ms
cypress:network:cors Parsed URL { port: '52701', tld: 'localhost', domain: '' } +384ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:52701', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:52701 +384ms
cypress:server:routes proxying static assets /__/assets/electron.fb07f5cc.svg, params[0] electron.fb07f5cc.svg +567ms
cypress:server:project socket:connected +684ms
cypress:server:run got socket connection { id: 'rt3xk1lond' } +1s
cypress:server:run socket connected { socketId: 'rt3xk1lond' } +0ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/vendors~main.js +2s
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/main.js +1ms
cypress:server:server-base Got CONNECT request from 127.0.0.1:8080 +777ms
cypress:network:connect successfully connected { opts: { port: 52701, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +779ms
cypress:server:util:socket_allowed allowing socket { localPort: 52816 } +778ms
cypress:server:server-base Got UPGRADE request from /ws +1ms
cypress:network:cors Parsed URL { port: '8080', tld: '127.0.0.1', domain: '' } +738ms
cypress:network:agent addRequest called { isHttps: false, href: 'http://127.0.0.1:8080/ws' } +3s
cypress:network:agent got family { family: 4, href: 'http://127.0.0.1:8080/ws' } +0ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/0.js +46ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/2.js +1ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/1.js +1ms
cypress:server:browsers:electron console message: { column: 22, level: 'info', line: 551, source: 'console-api', text: '[webpack-dev-server] Live Reloading enabled.', url: 'webpack:///(webpack)-dev-server/client/modules/logger/index.js?' } +1s
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/5.js +2ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/vendors~spec-0~spec-1~spec-10~spec-11~spec-12~spec-13~spec-14~spec-15~spec-16~spec-17~spec-18~spec-1~d37bff5e.js +240ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/spec-0~spec-1~spec-10~spec-11~spec-12~spec-13~spec-14~spec-15~spec-16~spec-17~spec-18~spec-19~spec-2~9d1ea79e.js +1ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/spec-2.js +1ms
cypress:server:project received runnables { id: 'r1', title: '', root: true, type: 'suite', file: 'cypress/component/Autocomplete.spec.tsx', retries: -1, _slow: 250, hooks: [ { title: '"before all" hook', hookName: 'before all', hookId: 'h1', body: "() => {\n cy.on('window:load', takeDOMSnapshot.init);\n}", type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"before all" hook', hookName: 'before all', hookId: 'h2', body: '() => {\n' + ' // we need to reset the coverage when running\n' + ' // in the interactive mode, otherwise the counters will\n' + ' // keep increasing every time we rerun the tests\n' + ' const logInstance = Cypress.log({\n' + " name: 'Coverage',\n" + " message: ['Reset [@cypress/code-coverage]']\n" + ' })\n' + '\n' + ' cy.task(\n' + " 'resetCoverage',\n" + ' {\n' + ' // @ts-ignore\n' + " isInteractive: Cypress.config('isInteractive')\n" + ' },\n' + ' { log: false }\n' + ' ).then(() => {\n' + ' logInstance.end()\n' + ' })\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"before each" hook', hookName: 'before each', hookId: 'h3', body: '() => {\n' + ' // each object will have the coverage and url pathname\n' + ' // to let the user know the coverage has been collected\n' + ' windowCoverageObjects = []\n' + '\n' + ' const saveCoverageObject = (win) => {\n' + ' // if application code has been instrumented, the app iframe "window" has an object\n' + ' const applicationSourceCoverage = win.__coverage__\n' + ' if (!applicationSourceCoverage) {\n' + ' return\n' + ' }\n' + '\n' + ' if (\n' + ' Cypress._.find(windowCoverageObjects, {\n' + ' coverage: applicationSourceCoverage\n' + ' })\n' + ' ) {\n' + ' // this application code coverage object is already known\n' + ' // which can happen when combining `window:load` and `before` callbacks\n' + ' return\n' + ' }\n' + '\n' + ' windowCoverageObjects.push({\n' + ' coverage: applicationSourceCoverage,\n' + ' pathname: win.location.pathname\n' + ' })\n' + ' }\n' + '\n' + ' // save reference to coverage for each app window loaded in the test\n' + " cy.on('window:load', saveCoverageObject)\n" + '\n' + ' // save reference if visiting a page inside a before() hook\n' + ' cy.window({ log: false }).then(saveCoverageObject)\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"after all" hook: collectBackendCoverage', hookName: 'after all', hookId: 'h4', body: 'function collectBackendCoverage() {\n' + ' // I wish I could fail the tests if there is no code coverage information\n' + ' // but throwing an error here does not fail the test run due to\n' + ' // https://github.com/cypress-io/cypress/issues/2296\n' + '\n' + ' // there might be server-side code coverage information\n' + ' // we should grab it once after all tests finish\n' + ' // @ts-ignore\n' + " const baseUrl = Cypress.config('baseUrl') || cy.state('window').origin\n" + ' // @ts-ignore\n' + " const runningEndToEndTests = baseUrl !== Cypress.config('proxyUrl')\n" + " const specType = Cypress._.get(Cypress.spec, 'specType', 'integration')\n" + " const isIntegrationSpec = specType === 'integration'\n" + '\n' + ' if (runningEndToEndTests && isIntegrationSpec) {\n' + ' // we can only request server-side code coverage\n' + ' // if we are running end-to-end tests,\n' + ' // otherwise where do we send the request?\n' + ' const url = Cypress._.get(\n' + " Cypress.env('codeCoverage'),\n" + " 'url',\n" + " '/__coverage__'\n" + ' )\n' + ' cy.request({\n' + ' url,\n' + ' log: false,\n' + ' failOnStatusCode: false\n' + ' })\n' + ' .then((r) => {\n' + " return Cypress._.get(r, 'body.coverage', null)\n" + ' })\n' + ' .then((coverage) => {\n' + ' if (!coverage) {\n' + ' // we did not get code coverage - this is the\n' + ' // original failed request\n' + ' const expectBackendCoverageOnly = Cypress._.get(\n' + " Cypress.env('codeCoverage'),\n" + " 'expectBackendCoverageOnly',\n" + ' false\n' + ' )\n' + ' if (expectBackendCoverageOnly) {\n' + ' throw new Error(\n' + ' `Expected to collect backend code coverage from ${url}`\n' + ' )\n' + ' } else {\n' + ' // we did not really expect to collect the backend code coverage\n' + ' return\n' + ' }\n' + ' }\n' + " sendCoverage(coverage, 'backend')\n" + ' })\n' + ' }\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"after all" hook: mergeUnitTestCoverage', hookName: 'after all', hookId: 'h5', body: 'function mergeUnitTestCoverage() {\n' + ' // collect and merge frontend coverage\n' + '\n' + ' // if spec bundle has been instrumented (using Cypress preprocessor)\n' + ' // then we will have unit test coverage\n' + ' // NOTE: spec iframe is NOT reset between the tests, so we can grab\n' + ' // the coverage information only once after all tests have finished\n' + ' // @ts-ignore\n' + ' const unitTestCoverage = window.__coverage__\n' + ' if (unitTestCoverage) {\n' + " sendCoverage(unitTestCoverage, 'unit')\n" + ' }\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"after all" hook: generateReport', hookName: 'after all', hookId: 'h6', body: 'function generateReport() {\n' + ' // when all tests finish, lets generate the coverage report\n' + ' const logInstance = Cypress.log({\n' + " name: 'Coverage',\n" + " message: ['Generating report [@cypress/code-coverage]']\n" + ' })\n' + " cy.task('coverageReport', null, {\n" + " timeout: dayjs.duration(3, 'minutes').asMilliseconds(),\n" + ' log: false\n' + ' }).then((coverageReportFolder) => {\n' + " logInstance.set('consoleProps', () => ({\n" + " 'coverage report folder': coverageReportFolder\n" + ' }))\n' + ' logInstance.end()\n' + ' return coverageReportFolder\n' + ' })\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"after each" hook', hookName: 'after each', hookId: 'h7', body: 'function () {\n cleanup();\n }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 }, { title: '"after each" hook', hookName: 'after each', hookId: 'h8', body: '() => {\n' + ' // save coverage after the test\n' + ' // because now the window coverage objects have been updated\n' + ' windowCoverageObjects.forEach((cover) => {\n' + ' sendCoverage(cover.coverage, cover.pathname)\n' + ' })\n' + '\n' + ' if (!hasE2ECoverage()) {\n' + ' if (hasUnitTestCoverage()) {\n' + ' logMessage(`👉 Only found unit test code coverage.`)\n' + ' } else {\n' + ' const expectBackendCoverageOnly = Cypress._.get(\n' + " Cypress.env('codeCoverage'),\n" + " 'expectBackendCoverageOnly',\n" + ' false\n' + ' )\n' + ' if (!expectBackendCoverageOnly) {\n' + ' logMessage(`\n' + ' ⚠️ Could not find any coverage information in your application\n' + ' by looking at the window coverage object.\n' + ' Did you forget to instrument your application?\n' + ' See [code-coverage#instrument-your-application](https://github.com/cypress-io/code-coverage#instrument-your-application)\n' + ' `)\n' + ' }\n' + ' }\n' + ' }\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250 } ], tests: [], suites: [ { id: 'r2', title: 'Autocomplete', root: false, type: 'suite', file: null, invocationDetails: [Object], retries: -1, _slow: 250, hooks: [Array], tests: [Array], suites: [] } ], runtimeConfig: {} } +714ms
cypress:server:reporter trying to load reporter: spec +2s
cypress:server:reporter spec is Mocha reporter +0ms
cypress:server:project onMocha start +7ms
cypress:server:reporter got mocha event 'start' with args: [ { start: '2022-06-22T20:53:29.113Z' } ] +5ms
cypress:server:project onMocha suite +4ms
cypress:server:reporter got mocha event 'suite' with args: [ { id: 'r1', title: '', root: true, type: 'suite', file: 'cypress/component/Autocomplete.spec.tsx', retries: -1, _slow: 250 } ] +4ms
cypress:server:project onMocha hook +39ms
cypress:server:reporter got mocha event 'hook' with args: [ { id: 'r3', title: '"before all" hook', hookName: 'before all', hookId: 'h1', body: "() => {\n cy.on('window:load', takeDOMSnapshot.init);\n}", type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite._createHook (http://localhost:8080/__cypress/runner/cypress_runner.js:176856:98)\n' + ' at Suite.../driver/node_modules/mocha/lib/suite.js.Suite.beforeAll (http://localhost:8080/__cypress/runner/cypress_runner.js:123812:19)\n' + ' at Suite.<computed> [as beforeAll] (http://localhost:8080/__cypress/runner/cypress_runner.js:176872:23)\n' + ' at before (http://localhost:8080/__cypress/runner/cypress_runner.js:117736:17)\n' + ' at eval (webpack:///./node_modules/happo-cypress/index.js?:4:1)\n' + ' at Object../node_modules/happo-cypress/index.js (http://localhost:8080/__cypress/src/2.js:3347:1)\n' + ' at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:85:30)\n' + ' at eval (webpack:///./cypress/support/component.ts?:2:71)\n' + ' at Module../cypress/support/component.ts (http://localhost:8080/__cypress/src/5.js:23:1)' }, currentRetry: 0, retries: -1, _slow: 250 } ] +39ms
cypress:server:project onMocha test:before:run +0ms
cypress:server:reporter got mocha event 'test:before:run' with args: [ { _testConfig: { testConfigList: [], unverifiedTestConfig: {} }, id: 'r3', order: 1, title: 'renders', body: '() => {\n' + ' cov_2hkpj1t1fa().f[14]++;\n' + ' cov_2hkpj1t1fa().s[42]++;\n' + ' cy.mount( /*#__PURE__*/react__WEBPACK_IMPORTED_MODULE_0___default.a.createElement(TestAutocomplete, null));\n' + ' cov_2hkpj1t1fa().s[43]++;\n' + " cy.get('body').happoScreenshot();\n" + ' }', type: 'test', wallClockStartedAt: '2022-06-22T20:53:29.119Z', file: null, invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite.addTest (http://localhost:8080/__cypress/runner/cypress_runner.js:176767:93)\n' + ' at context.it.context.specify (http://localhost:8080/__cypress/runner/cypress_runner.js:117646:13)\n' + ' at http://localhost:8080/__cypress/runner/cypress_runner.js:176520:21\n' + ' at Suite.eval (webpack:///./cypress/component/Autocomplete.spec.tsx?:2437:3)\n' + ' at Object.create (http://localhost:8080/__cypress/runner/cypress_runner.js:117823:19)\n' + ' at context.describe.context.context (http://localhost:8080/__cypress/runner/cypress_runner.js:117599:27)\n' + ' at http://localhost:8080/__cypress/runner/cypress_runner.js:176520:21\n' + ' at eval (webpack:///./cypress/component/Autocomplete.spec.tsx?:2434:1)\n' + ' at Module../cypress/component/Autocomplete.spec.tsx (http://localhost:8080/__cypress/src/spec-2.js:11:1)' }, currentRetry: 0, retries: 0, _slow: 250 } ] +0ms
cypress:server:socket-base automation:request get:cookies { domain: 'localhost' } +2s
cypress:server:automation:cookies getting:cookies { domain: 'localhost' } +2s
cypress:server:socket-base backend:request { eventName: 'reset:server:state', args: [] } +0ms
cypress:server:remote-states resetting remote state +1s
cypress:server:automation:cookies received get:cookies [] +1ms
cypress:server:project onMocha hook end +5ms
cypress:server:reporter got mocha event 'hook end' with args: [ { id: 'r3', title: '"before all" hook', hookName: 'before all', hookId: 'h1', body: "() => {\n cy.on('window:load', takeDOMSnapshot.init);\n}", type: 'hook', duration: 43, file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite._createHook (http://localhost:8080/__cypress/runner/cypress_runner.js:176856:98)\n' + ' at Suite.../driver/node_modules/mocha/lib/suite.js.Suite.beforeAll (http://localhost:8080/__cypress/runner/cypress_runner.js:123812:19)\n' + ' at Suite.<computed> [as beforeAll] (http://localhost:8080/__cypress/runner/cypress_runner.js:176872:23)\n' + ' at before (http://localhost:8080/__cypress/runner/cypress_runner.js:117736:17)\n' + ' at eval (webpack:///./node_modules/happo-cypress/index.js?:4:1)\n' + ' at Object../node_modules/happo-cypress/index.js (http://localhost:8080/__cypress/src/2.js:3347:1)\n' + ' at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:85:30)\n' + ' at eval (webpack:///./cypress/support/component.ts?:2:71)\n' + ' at Module../cypress/support/component.ts (http://localhost:8080/__cypress/src/5.js:23:1)' }, currentRetry: 0, retries: -1, _slow: 250 } ] +5ms
cypress:server:project onMocha hook +5ms
cypress:server:reporter got mocha event 'hook' with args: [ { id: 'r3', title: '"before all" hook', hookName: 'before all', hookId: 'h2', body: '() => {\n' + ' // we need to reset the coverage when running\n' + ' // in the interactive mode, otherwise the counters will\n' + ' // keep increasing every time we rerun the tests\n' + ' const logInstance = Cypress.log({\n' + " name: 'Coverage',\n" + " message: ['Reset [@cypress/code-coverage]']\n" + ' })\n' + '\n' + ' cy.task(\n' + " 'resetCoverage',\n" + ' {\n' + ' // @ts-ignore\n' + " isInteractive: Cypress.config('isInteractive')\n" + ' },\n' + ' { log: false }\n' + ' ).then(() => {\n' + ' logInstance.end()\n' + ' })\n' + ' }', type: 'hook', file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite._createHook (http://localhost:8080/__cypress/runner/cypress_runner.js:176856:98)\n' + ' at Suite.../driver/node_modules/mocha/lib/suite.js.Suite.beforeAll (http://localhost:8080/__cypress/runner/cypress_runner.js:123812:19)\n' + ' at Suite.<computed> [as beforeAll] (http://localhost:8080/__cypress/runner/cypress_runner.js:176872:23)\n' + ' at before (http://localhost:8080/__cypress/runner/cypress_runner.js:117736:17)\n' + ' at registerHooks (webpack:///./node_modules/@cypress/code-coverage/support.js?:42:3)\n' + ' at eval (webpack:///./node_modules/@cypress/code-coverage/support.js?:237:3)\n' + ' at Object../node_modules/@cypress/code-coverage/support.js (http://localhost:8080/__cypress/src/2.js:44:1)\n' + ' at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:85:30)\n' + ' at eval (webpack:///./node_modules/@toptal/davinci-qa/src/configs/cypress/support/index.js?:2:88)' }, currentRetry: 0, retries: -1, _slow: 250 } ] +5ms
cypress:server:socket-base backend:request { eventName: 'task', args: [ { task: 'resetCoverage', arg: [Object], timeout: 60000 } ] } +9ms
cypress:server:task run task resetCoverage with arg { isInteractive: false } +2s
cypress:server:task result is: null +1ms
cypress:server:project onMocha hook end +3ms
cypress:server:reporter got mocha event 'hook end' with args: [ { id: 'r3', title: '"before all" hook', hookName: 'before all', hookId: 'h2', body: '() => {\n' + ' // we need to reset the coverage when running\n' + ' // in the interactive mode, otherwise the counters will\n' + ' // keep increasing every time we rerun the tests\n' + ' const logInstance = Cypress.log({\n' + " name: 'Coverage',\n" + " message: ['Reset [@cypress/code-coverage]']\n" + ' })\n' + '\n' + ' cy.task(\n' + " 'resetCoverage',\n" + ' {\n' + ' // @ts-ignore\n' + " isInteractive: Cypress.config('isInteractive')\n" + ' },\n' + ' { log: false }\n' + ' ).then(() => {\n' + ' logInstance.end()\n' + ' })\n' + ' }', type: 'hook', duration: 8, file: 'cypress/component/Autocomplete.spec.tsx', invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite._createHook (http://localhost:8080/__cypress/runner/cypress_runner.js:176856:98)\n' + ' at Suite.../driver/node_modules/mocha/lib/suite.js.Suite.beforeAll (http://localhost:8080/__cypress/runner/cypress_runner.js:123812:19)\n' + ' at Suite.<computed> [as beforeAll] (http://localhost:8080/__cypress/runner/cypress_runner.js:176872:23)\n' + ' at before (http://localhost:8080/__cypress/runner/cypress_runner.js:117736:17)\n' + ' at registerHooks (webpack:///./node_modules/@cypress/code-coverage/support.js?:42:3)\n' + ' at eval (webpack:///./node_modules/@cypress/code-coverage/support.js?:237:3)\n' + ' at Object../node_modules/@cypress/code-coverage/support.js (http://localhost:8080/__cypress/src/2.js:44:1)\n' + ' at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:85:30)\n' + ' at eval (webpack:///./node_modules/@toptal/davinci-qa/src/configs/cypress/support/index.js?:2:88)' }, currentRetry: 0, retries: -1, _slow: 250 } ] +3ms
cypress:server:project onMocha suite +2ms
cypress:server:reporter got mocha event 'suite' with args: [ { id: 'r2', title: 'Autocomplete', root: false, type: 'suite', file: null, invocationDetails: { function: 'Object.getInvocationDetails', fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js', originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js', line: 180642, column: 18, whitespace: ' ', stack: 'Error\n' + ' at Object.getInvocationDetails (http://localhost:8080/__cypress/runner/cypress_runner.js:180642:17)\n' + ' at Suite.addSuite (http://localhost:8080/__cypress/runner/cypress_runner.js:176799:94)\n' + ' at Function.../driver/node_modules/mocha/lib/suite.js.Suite.create (http://localhost:8080/__cypress/runner/cypress_runner.js:123595:10)\n' + ' at Object.create (http://localhost:8080/__cypress/runner/cypress_runner.js:117806:27)\n' + ' at context.describe.context.context (http://localhost:8080/__cypress/runner/cypress_runner.js:117599:27)\n' + ' at http://localhost:8080/__cypress/runner/cypress_runner.js:176520:21\n' + ' at eval (webpack:///./cypress/component/Autocomplete.spec.tsx?:2434:1)\n' + ' at Module../cypress/component/Autocomplete.spec.tsx (http://localhost:8080/__cypress/src/spec-2.js:11:1)\n' + ' at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:85:30)' }, retries: -1, _slow: 250 } ] +2ms
Autocomplete
Same here. Almost double as slow.
On v9 my 200+ component tests would run on Github Actions in 8m 24s On v10 the same number of component tests now runs in 14m 30s
Sometimes it even goes over 20min, so I had to increase the timeout from 20min to 30min for now...
I am using run --component --browser=chrome, otherwise with electron I would run in the bug reported here https://github.com/cypress-io/cypress/issues/22208
Maybe dup/related? https://github.com/cypress-io/cypress/issues/22208
I am looking into this. Performance is key, fixing this is the highest priority.
I'll put my notes here. I ran these on the branches cypress-9 and cypress-10 in this repo: https://github.com/lmiller1990/cra-cypresss-lots-of-specs/actions on my local machine. It does the same trivial spec 100 times. You can also see the runs in CI. I ran them headlessly in chrome.
Cypress 9 - 175s. Cypress 10 - 226s.
Cypress 9 CT: Average time per spec in ms 859ms Cypress 10 CT: Average time per spec in ms 984ms
There is a delta between my two repos of 51 seconds - so, we spent 51 seconds doing (some extra thing) in Cypress 10 for component testing. I will find out what. About 12 of those seconds are accounted for in the actual test spec execution, and the remaining 39 in (something). I'm guessing the actual test execution time is likely to be identical, since we inject the same code (your test and components). The ~125ms is likely also caused by whatever is causing the major slowdown.
Edit: Some additional logging from Cypress 10 we don't have in 9, might unearth some changes. I'm kind of thinking out loud here. One change in Cypress 10 for component testing is a new tab per spec, to better improve isolation between specs. E2E always did this - in fact, it opened an entire browser per spec, so this pref regression would be only noticable for component testing.
Looking at some logs, the main things that stick out:
cypress:server:open_project resetting project state,
cypress:server:project resetting project instance
cypress:server:open_project launching browser
cypress:server:browsers getBrowserLauncher
cypress:server:browsers:chrome connecting
cypress:server:browsers:cri-client registering CDP on event
cypress:server:browsers:chrome starting screencast
cypress:server:browsers:chrome received CRI client
cypress:server:browsers:chrome navigating to page
cypress:server:server-base
cypress:server:server-base
cypress:server:routes proxying static assets
cypress:https-proxy Writing browserSocket connection headers
cypress:server:server-base Got UPGRADE request from /__socket-graphql
I'll dive deeper, but I'm pretty sure the main cost is the new tab per spec, which also means we have some overhead with
- serving runner static assets
- reconnecting the websocket
I'm not sure what resetting a project instance does exactly, I'll find out. Also, we use GraphQL in the open mode UI, but not run mode - it should not be starting or even making an appearance, but I think we have a web socket that is erronously starting up.
Lots of logs
## Cypress 9We proceed from video capture -> after:spec -> ending the video recording -> Started processing.
cypress:server:video capture ended +0ms
cypress:server:plugins plugin event registered? { event: 'after:spec', isRegistered: false } +3s
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 1 second │
│ Spec Ran: App_90.cy.jsx │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
cypress:server:run ending the video recording { name: '/home/lachlan/code/dump/lots-of-specs-9/cypress/videos/App_90.cy.jsx.mp4', videoCompression: 32, shouldUploadVideo: true } +2s
(Video)
- Started processing: Compressing to 32 CRF
cypress:server:video processing video from /home/lachlan/code/dump/lots-of-specs-9/cypress/videos/App_90.cy.jsx.mp4 to /home/lachlan/code/dump/lots-of-specs-9/cypress/videos/App_90.cy.jsx-compressed.mp4 video compression 32 +4ms
Cypress 10:
We proceed from video capture -> after:spec -> close browser tab -> ending the video recording -> Started processing.
We have this additional logging in between:
cypress:server:run attempting to close the browser tab +2s
cypress:server:browsers:browser-cri-client Closing current target 92CCF585E9E37F5D53DE6F62B6AF364F +3s
cypress:server:browsers:cri-client connecting { target: 'C785A341B53A8EBD6D864807E1772D77' } +3s
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 42150 } +2s
Full log:
cypress:lifecycle:EventRegistrar plugin event registered? { eventName: 'after:spec', isRegistered: false } +3s
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 1 second │
│ Spec Ran: App_90.cy.jsx │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
cypress:server:run attempting to close the browser tab +2s
cypress:server:browsers:browser-cri-client Closing current target 92CCF585E9E37F5D53DE6F62B6AF364F +3s
cypress:server:browsers:cri-client connecting { target: 'C785A341B53A8EBD6D864807E1772D77' } +3s
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 42150 } +2s
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 42134 } +1ms
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 42146 } +0ms
cypress:server:socket-base socket-disconnecting transport close +2s
cypress:server:socket-base socket-disconnect transport close +0ms
cypress:server:run resetting server state +22ms
cypress:proxy:http:util:buffers resetting buffers +2s
cypress:network:cors Parsed URL { port: '80', tld: '', domain: '' } +2s
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +0ms
cypress:server:remote-states setting remote state { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for http://localhost:37763 +2s
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +0ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:37763 +0ms
cypress:server:run ending the video recording { name: '/home/lachlan/code/dump/lots-of-specs-10/cypress/videos/App_90.cy.jsx.mp4', videoCompression: 32, shouldUploadVideo: true } +2ms
(Video)
- Started processing: Compressing to 32 CRF
cypress:server:video processing video from /home/lachlan/code/dump/lots-of-specs-10/cypress/videos/App_90.cy.jsx.mp4 to /home/lachlan/code/dump/lots-of-specs-10/cypress/videos/App_90.cy.jsx-compressed.mp4 video compression 32 +30ms
Between Specs Logs
Cypress 9
Running: App_91.cy.jsx (2 of 2)
cypress:server:run about to run spec { spec: { name: 'App_91.cy.jsx', relative: 'src/App_91.cy.jsx', absolute: '/home/lachlan/code/dump/lots-of-specs-9/src/App_91.cy.jsx', specType: 'component' }, isHeadless: true, browser: { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103, isHeadless: true, isHeaded: false } } +1ms
cypress:server:plugins plugin event registered? { event: 'before:spec', isRegistered: false } +417ms
cypress:server:run video recording has been enabled. video: true +0ms
cypress:server:video capture started { command: "ffmpeg -n 20 /home/lachlan/.cache/Cypress/9.7.0/Cypress/resources/app/node_modules/@ffmpeg-installer/linux-x64/ffmpeg -f image2pipe -use_wallclock_as_timestamps 1 -i pipe:0 -y -vcodec libx264 -filter:v crop='floor(in_w/2)*2:floor(in_h/2)*2' -preset ultrafast /home/lachlan/code/dump/lots-of-specs-9/cypress/videos/App_91.cy.jsx.mp4" } +18ms
cypress:server:run waiting for socket to connect and browser to launch... +12ms
cypress:server:run navigating to next spec +1ms
cypress:server:project_utils get spec url: /home/lachlan/code/dump/lots-of-specs-9/src/App_91.cy.jsx for spec type component +4s
cypress:server:project_utils prefixed path for spec { pathToSpec: '/home/lachlan/code/dump/lots-of-specs-9/src/App_91.cy.jsx', type: 'component', url: '/component/App_91.cy.jsx' } +0ms
cypress:server:project_utils return path to spec { specType: 'component', absoluteSpecPath: '/home/lachlan/code/dump/lots-of-specs-9/src/App_91.cy.jsx', prefixedPath: '/component/App_91.cy.jsx', url: 'http://localhost:8080/__/#/tests/component/App_91.cy.jsx' } +0ms
cypress:server:video capture stderr log { message: 'ffmpeg version N-47683-g0e8eb07980-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2018 the FFmpeg developers' } +1ms
cypress:server:video capture stderr log { message: ' built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516' } +1ms
cypress:server:video capture stderr log { message: ' configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg' } +0ms
cypress:server:video capture stderr log { message: ' libavutil 56. 24.101 / 56. 24.101' } +0ms
cypress:server:video capture stderr log { message: ' libavcodec 58. 42.100 / 58. 42.100' } +0ms
cypress:server:video capture stderr log { message: ' libavformat 58. 24.100 / 58. 24.100' } +0ms
cypress:server:video capture stderr log { message: ' libavdevice 58. 6.101 / 58. 6.101' } +0ms
cypress:server:video capture stderr log { message: ' libavfilter 7. 46.101 / 7. 46.101' } +0ms
cypress:server:video capture stderr log { message: ' libswscale 5. 4.100 / 5. 4.100' } +0ms
cypress:server:video capture stderr log { message: ' libswresample 3. 4.100 / 3. 4.100' } +0ms
cypress:server:video capture stderr log { message: ' libpostproc 55. 4.100 / 55. 4.100' } +0ms
cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 53722 } +3s
cypress:server:server-base Got CONNECT request from localhost:8080 +3s
cypress:https-proxy Writing browserSocket connection headers { url: 'localhost:8080', headLength: 0, headers: { host: 'localhost:8080', 'proxy-connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/103.0.5060.53 Safari/537.36' } } +3s
cypress:https-proxy Got first head bytes { url: 'localhost:8080', head: 'GET /ws HTTP/1.1\r\nHost: localhost:8080\r\nConnection: Upgrade\r\nPra' } +0ms
cypress:https-proxy Making intercepted connection to 41055 +0ms
cypress:network:connect successfully connected { opts: { port: 41055, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +3s
cypress:https-proxy received upstreamSocket callback for request { port: 41055, hostname: 'localhost', err: undefined } +1ms
cypress:server:util:socket_allowed allowing socket { localPort: 53734 } +44ms
cypress:server:server-base Got UPGRADE request from /ws +2ms
cypress:network:cors Parsed URL { port: '8080', tld: 'localhost', domain: '' } +3s
cypress:network:agent addRequest called { isHttps: false, href: 'http://localhost:8080/ws' } +3s
cypress:network:agent got family { family: 4, href: 'http://localhost:8080/ws' } +0ms
cypress:server:project received runnables { id: 'r1', title: '', root: true, type: 'suite', file: 'src/App_91.cy.jsx', retries: -1, _slow: 250, hooks: [], tests: [ { id: 'r2', title: '[App_91.cy.jsx]: renders learn react link', body: '() => {\n' + ' (0,_cypress_react__WEBPACK_IMPORTED_MODULE_1__.mount)( /*#__PURE__*/(0,react_jsx_dev_runtime__WEBPACK_IMPORTED_MODULE_2__.jsxDEV)(_App__WEBPACK_IMPORTED_MODULE_0__["default"], {}, void 0, false, {\n' + ' fileName: _jsxFileName,\n' + ' lineNumber: 5,\n' + ' columnNumber: 9\n' + ' }, undefined));\n' + ' cy.get("a").contains("React");\n' + '\n' + ' for (let i = 1; i < 10; i++) {\n' + ' cy.get("button").click();\n' + ' cy.get("p").contains(i);\n' + ' }\n' + '}', type: 'test', file: null, invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250, hooks: [Array], _testConfig: [Object], _titlePath: [Array] } ], suites: [], runtimeConfig: {} } +2s
cypress:server:reporter trying to load reporter: spec +2s
cypress:server:reporter spec is Mocha reporter +0ms
cypress:server:project onMocha start +3ms
Cypress 10
cypress:server:run about to run spec { spec: { fileExtension: '.jsx', baseName: 'App_91.cy.jsx', fileName: 'App_91', specFileExtension: '.cy.jsx', relativeToCommonRoot: 'App_91.cy.jsx', specType: 'component', name: 'src/App_91.cy.jsx', relative: 'src/App_91.cy.jsx', absolute: '/home/lachlan/code/dump/lots-of-specs-10/src/App_91.cy.jsx' }, isHeadless: true, browser: { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103, isHeadless: true, isHeaded: false } } +1ms
cypress:lifecycle:EventRegistrar plugin event registered? { eventName: 'before:spec', isRegistered: false } +401ms
cypress:server:run video recording has been enabled. video: true +1ms
cypress:server:video capture started { command: "ffmpeg -n 20 /home/lachlan/.cache/Cypress/10.3.0/Cypress/resources/app/node_modules/@ffmpeg-installer/linux-x64/ffmpeg -f image2pipe -use_wallclock_as_timestamps 1 -i pipe:0 -y -vcodec libx264 -filter:v crop='floor(in_w/2)*2:floor(in_h/2)*2' -preset ultrafast /home/lachlan/code/dump/lots-of-specs-10/cypress/videos/App_91.cy.jsx.mp4" } +14ms
cypress:server:run waiting for socket to connect and browser to launch... +10ms
cypress:server:run waiting for socket connection... { id: 'qcjyc5ci34' } +0ms
cypress:server:run setting Chrome properties { shouldWriteVideo: true } +0ms
cypress:server:run browser launched +1ms
cypress:server:open_project resetting project state, preparing to launch browser chrome for spec { fileExtension: '.jsx', baseName: 'App_91.cy.jsx', fileName: 'App_91', specFileExtension: '.cy.jsx', relativeToCommonRoot: 'App_91.cy.jsx', specType: 'component', name: 'src/App_91.cy.jsx', relative: 'src/App_91.cy.jsx', absolute: '/home/lachlan/code/dump/lots-of-specs-10/src/App_91.cy.jsx' } options { onScreencastFrame: [Function (anonymous)], automationMiddleware: { onBeforeRequest: [Function: onBeforeRequest], onAfterResponse: [Function: onAfterResponse] }, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', shouldLaunchNewTab: true, onWarning: [Function (anonymous)] } +4s
cypress:server:project resetting project instance /home/lachlan/code/dump/lots-of-specs-10 +2s
cypress:proxy:http:util:buffers resetting buffers +387ms
cypress:network:cors Parsed URL { port: '80', tld: '', domain: '' } +386ms
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +0ms
cypress:server:remote-states setting remote state { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for http://localhost:37763 +386ms
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +0ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:37763 +0ms
cypress:server:project_utils returning spec url http://localhost:8080/__/#/specs/runner?file=src/App_91.cy.jsx +4s
cypress:server:open_project open project url http://localhost:8080/__/#/specs/runner?file=src/App_91.cy.jsx +0ms
cypress:server:project project has config { devServer: { framework: 'create-react-app', bundler: 'webpack' }, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10', rawJson: { component: { devServer: [Object] }, devServer: { framework: 'create-react-app', bundler: 'webpack' }, envFile: {}, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10' }, configFile: 'cypress.config.js', morgan: false, isTextTerminal: true, socketId: 'qcjyc5ci34', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, chromeWebSecurity: true, clientCertificates: [], defaultCommandTimeout: 4000, downloadsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/downloads', env: {}, execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/home/lachlan/code/dump/lots-of-specs-10', fixturesFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/fixtures', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'linux', pageLoadTimeout: 60000, port: 37763, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/home/lachlan/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, screenshotsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/screenshots', slowTestThreshold: 250, scrollBehavior: 'top', supportFile: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support/component.js', supportFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, video: true, videoCompression: 32, videosFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/videos', videoUploadOnPasses: true, viewportHeight: 500, viewportWidth: 500, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '103.0.5060.53', path: 'chromium', profilePath: '/home/lachlan/snap/chromium/current', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: 'firefox', minSupportedVersion: 86, majorVersion: 101 }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.160', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/home/lachlan/.cache/Cypress/10.3.0/Cypress/resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.3.0', xhrRoute: '/xhrs/', specPattern: '**/*.cy.{js,jsx,ts,tsx}', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: true, from: 'default' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 4000, from: 'default' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: {}, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: 'cypress/fixtures', from: 'default' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'linux', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: 'cypress/screenshots', from: 'default' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,jsx,ts,tsx}', from: 'default' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: true, from: 'default' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 500, from: 'default' }, viewportWidth: { value: 500, from: 'default' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, specPattern: { value: '**/*.cy.{js,jsx,ts,tsx}', from: 'default' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, testingType: { value: 'component' } }, testingType: 'component', componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:37763', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:37763', state: {} } +0ms
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +1ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:37763 +1ms
cypress:server:open_project launching browser: { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103, isHeadless: true, isHeaded: false }, spec: src/App_91.cy.jsx +1ms
cypress:server:browsers getBrowserLauncher { browser: { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103, isHeadless: true, isHeaded: false } } +4s
cypress:server:browsers:chrome connecting to new chrome tab in existing instance with url and debugging port {
url: 'http://localhost:8080/__/#/specs/runner?file=src/App_91.cy.jsx'
} +4s
cypress:server:browsers:cri-client registering CDP on event { eventName: 'Network.requestWillBeSent' } +398ms
cypress:server:browsers:cri-client registering CDP on event { eventName: 'Network.responseReceived' } +0ms
cypress:server:browsers:chrome starting screencast +53ms
cypress:server:browsers:cri-client registering CDP on event { eventName: 'Page.screencastFrame' } +53ms
cypress:server:browsers:cri-client registering CDP on event { eventName: 'Page.downloadWillBegin' } +0ms
cypress:server:browsers:cri-client registering CDP on event { eventName: 'Page.downloadProgress' } +0ms
cypress:server:browsers:chrome received CRI client +3ms
cypress:server:browsers:chrome navigating to page http://localhost:8080/__/#/specs/runner?file=src/App_91.cy.jsx +0ms
cypress:server:project project has config { devServer: { framework: 'create-react-app', bundler: 'webpack' }, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10', rawJson: { component: { devServer: [Object] }, devServer: { framework: 'create-react-app', bundler: 'webpack' }, envFile: {}, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10' }, configFile: 'cypress.config.js', morgan: false, isTextTerminal: true, socketId: 'qcjyc5ci34', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, chromeWebSecurity: true, clientCertificates: [], defaultCommandTimeout: 4000, downloadsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/downloads', env: {}, execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/home/lachlan/code/dump/lots-of-specs-10', fixturesFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/fixtures', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'linux', pageLoadTimeout: 60000, port: 37763, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/home/lachlan/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, screenshotsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/screenshots', slowTestThreshold: 250, scrollBehavior: 'top', supportFile: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support/component.js', supportFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, video: true, videoCompression: 32, videosFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/videos', videoUploadOnPasses: true, viewportHeight: 500, viewportWidth: 500, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '103.0.5060.53', path: 'chromium', profilePath: '/home/lachlan/snap/chromium/current', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: 'firefox', minSupportedVersion: 86, majorVersion: 101 }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.160', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/home/lachlan/.cache/Cypress/10.3.0/Cypress/resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.3.0', xhrRoute: '/xhrs/', specPattern: '**/*.cy.{js,jsx,ts,tsx}', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: true, from: 'default' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 4000, from: 'default' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: {}, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: 'cypress/fixtures', from: 'default' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'linux', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: 'cypress/screenshots', from: 'default' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,jsx,ts,tsx}', from: 'default' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: true, from: 'default' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 500, from: 'default' }, viewportWidth: { value: 500, from: 'default' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, specPattern: { value: '**/*.cy.{js,jsx,ts,tsx}', from: 'default' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, testingType: { value: 'component' } }, testingType: 'component', componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:37763', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:37763', state: {} } +64ms
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +64ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:37763 +64ms
cypress:server:routes proxying static assets /__/assets/index.7f47beef.js, params[0] index.7f47beef.js +3s
cypress:server:routes proxying static assets /__/assets/index.20a7a5c6.css, params[0] index.20a7a5c6.css +1ms
cypress:server:server-base Got CONNECT request from localhost:8080 +3s
cypress:https-proxy Writing browserSocket connection headers { url: 'localhost:8080', headLength: 0, headers: { host: 'localhost:8080', 'proxy-connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/103.0.5060.53 Safari/537.36' } } +3s
cypress:https-proxy Got first head bytes { url: 'localhost:8080', head: 'GET /__socket/?EIO=4&transport=websocket HTTP/1.1\r\nHost: localho' } +2ms
cypress:https-proxy Making intercepted connection to 37763 +0ms
cypress:network:connect successfully connected { opts: { port: 37763, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +3s
cypress:https-proxy received upstreamSocket callback for request { port: 37763, hostname: 'localhost', err: undefined } +1ms
cypress:server:util:socket_allowed allowing socket { localPort: 42168 } +682ms
cypress:server:server-base Got UPGRADE request from /__socket/?EIO=4&transport=websocket +6ms
cypress:server:util:socket_allowed is incoming request allowed? { isAllowed: true, reqUrl: '/__socket/?EIO=4&transport=websocket', remotePort: 42168, remoteAddress: '127.0.0.1' } +2ms
cypress:server:socket-base socket connected +696ms
cypress:server:socket-ct do onSocketConnection +4s
cypress:server:routes proxying static assets /__/assets/Specs.3607dc55.js, params[0] Specs.3607dc55.js +258ms
cypress:server:routes proxying static assets /__/assets/Runner.f82e8a36.js, params[0] Runner.f82e8a36.js +1ms
cypress:server:routes proxying static assets /__/assets/Runner.04245e60.css, params[0] Runner.04245e60.css +0ms
cypress:server:routes proxying static assets /__/assets/CreateSpecModal.d142ed01.js, params[0] CreateSpecModal.d142ed01.js +1ms
cypress:server:routes proxying static assets /__/assets/SpecPatterns.38ca72aa.js, params[0] SpecPatterns.38ca72aa.js +0ms
cypress:server:routes proxying static assets /__/assets/add-large_x16.8ef42c8a.js, params[0] add-large_x16.8ef42c8a.js +7ms
cypress:server:routes proxying static assets /__/assets/Switch.139b3d04.js, params[0] Switch.139b3d04.js +6ms
cypress:server:routes proxying static assets /__/assets/refresh_x16.08be4859.js, params[0] refresh_x16.08be4859.js +0ms
cypress:server:server-base Got CONNECT request from localhost:8080 +130ms
cypress:https-proxy Writing browserSocket connection headers { url: 'localhost:8080', headLength: 0, headers: { host: 'localhost:8080', 'proxy-connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/103.0.5060.53 Safari/537.36' } } +132ms
cypress:https-proxy Got first head bytes { url: 'localhost:8080', head: 'GET /__socket-graphql HTTP/1.1\r\nHost: localhost:8080\r\nConnection' } +4ms
cypress:https-proxy Making intercepted connection to 37763 +0ms
cypress:network:connect successfully connected { opts: { port: 37763, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +138ms
cypress:https-proxy received upstreamSocket callback for request { port: 37763, hostname: 'localhost', err: undefined } +2ms
cypress:server:util:socket_allowed allowing socket { localPort: 42180 } +136ms
cypress:server:server-base Got UPGRADE request from /__socket-graphql +8ms
cypress:server:util:socket_allowed is incoming request allowed? { isAllowed: true, reqUrl: '/__socket-graphql', remotePort: 42180, remoteAddress: '127.0.0.1' } +2ms
cypress:server:project project has config { devServer: { framework: 'create-react-app', bundler: 'webpack' }, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10', rawJson: { component: { devServer: [Object] }, devServer: { framework: 'create-react-app', bundler: 'webpack' }, envFile: {}, projectRoot: '/home/lachlan/code/dump/lots-of-specs-10', projectName: 'lots-of-specs-10' }, configFile: 'cypress.config.js', morgan: false, isTextTerminal: true, socketId: 'qcjyc5ci34', report: true, animationDistanceThreshold: 5, arch: 'x64', baseUrl: 'http://localhost:8080', blockHosts: null, chromeWebSecurity: true, clientCertificates: [], defaultCommandTimeout: 4000, downloadsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/downloads', env: {}, execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalSessionAndOrigin: false, experimentalSourceRewriting: false, fileServerFolder: '/home/lachlan/code/dump/lots-of-specs-10', fixturesFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/fixtures', excludeSpecPattern: [ '**/__snapshots__/*', '**/__image_snapshots__/*' ], includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'linux', pageLoadTimeout: 60000, port: 37763, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/home/lachlan/.nvm/versions/node/v16.13.2/bin/node', resolvedNodeVersion: '16.13.2', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, screenshotsFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/screenshots', slowTestThreshold: 250, scrollBehavior: 'top', supportFile: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support/component.js', supportFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/support', taskTimeout: 60000, trashAssetsBeforeRuns: true, userAgent: null, video: true, videoCompression: 32, videosFolder: '/home/lachlan/code/dump/lots-of-specs-10/cypress/videos', videoUploadOnPasses: true, viewportHeight: 500, viewportWidth: 500, waitForAnimations: true, watchForFileChanges: false, additionalIgnorePattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}', autoOpen: false, browsers: [ { name: 'chrome', family: 'chromium', channel: 'stable', displayName: 'Chrome', version: '103.0.5060.53', path: 'google-chrome', minSupportedVersion: 64, majorVersion: 103 }, { name: 'chromium', family: 'chromium', channel: 'stable', displayName: 'Chromium', version: '103.0.5060.53', path: 'chromium', profilePath: '/home/lachlan/snap/chromium/current', minSupportedVersion: 64, majorVersion: 103 }, { name: 'firefox', family: 'firefox', channel: 'stable', displayName: 'Firefox', version: '101.0.1', path: 'firefox', minSupportedVersion: 86, majorVersion: 101 }, { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.160', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } ], clientRoute: '/__/', cypressBinaryRoot: '/home/lachlan/.cache/Cypress/10.3.0/Cypress/resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '10.3.0', xhrRoute: '/xhrs/', specPattern: '**/*.cy.{js,jsx,ts,tsx}', indexHtmlFile: 'cypress/support/component-index.html', cypressEnv: 'production', resolved: { animationDistanceThreshold: { value: 5, from: 'default' }, arch: { value: 'x64', from: 'default' }, baseUrl: { value: null, from: 'default' }, blockHosts: { value: null, from: 'default' }, chromeWebSecurity: { value: true, from: 'default' }, clientCertificates: { value: [], from: 'default' }, defaultCommandTimeout: { value: 4000, from: 'default' }, downloadsFolder: { value: 'cypress/downloads', from: 'default' }, env: {}, execTimeout: { value: 60000, from: 'default' }, experimentalFetchPolyfill: { value: false, from: 'default' }, experimentalInteractiveRunEvents: { value: false, from: 'default' }, experimentalSessionAndOrigin: { value: false, from: 'default' }, experimentalSourceRewriting: { value: false, from: 'default' }, fileServerFolder: { value: '', from: 'default' }, fixturesFolder: { value: 'cypress/fixtures', from: 'default' }, excludeSpecPattern: { value: [Array], from: 'default' }, includeShadowDom: { value: false, from: 'default' }, keystrokeDelay: { value: 0, from: 'default' }, modifyObstructiveCode: { value: true, from: 'default' }, numTestsKeptInMemory: { value: 0, from: 'config' }, platform: { value: 'linux', from: 'default' }, pageLoadTimeout: { value: 60000, from: 'default' }, port: { value: null, from: 'default' }, projectId: { value: null, from: 'default' }, redirectionLimit: { value: 20, from: 'default' }, reporter: { value: 'spec', from: 'default' }, reporterOptions: { value: null, from: 'default' }, requestTimeout: { value: 5000, from: 'default' }, resolvedNodePath: { value: null, from: 'default' }, resolvedNodeVersion: { value: null, from: 'default' }, responseTimeout: { value: 30000, from: 'default' }, retries: { value: [Object], from: 'default' }, screenshotOnRunFailure: { value: true, from: 'default' }, screenshotsFolder: { value: 'cypress/screenshots', from: 'default' }, slowTestThreshold: { value: 250, from: 'default' }, scrollBehavior: { value: 'top', from: 'default' }, supportFile: { value: 'cypress/support/component.{js,jsx,ts,tsx}', from: 'default' }, supportFolder: { value: false, from: 'default' }, taskTimeout: { value: 60000, from: 'default' }, trashAssetsBeforeRuns: { value: true, from: 'default' }, userAgent: { value: null, from: 'default' }, video: { value: true, from: 'default' }, videoCompression: { value: 32, from: 'default' }, videosFolder: { value: 'cypress/videos', from: 'default' }, videoUploadOnPasses: { value: true, from: 'default' }, viewportHeight: { value: 500, from: 'default' }, viewportWidth: { value: 500, from: 'default' }, waitForAnimations: { value: true, from: 'default' }, watchForFileChanges: { value: false, from: 'config' }, specPattern: { value: '**/*.cy.{js,jsx,ts,tsx}', from: 'default' }, browsers: { value: [Array], from: 'runtime' }, hosts: { value: null, from: 'default' }, isInteractive: { value: true, from: 'default' }, testingType: { value: 'component' } }, testingType: 'component', componentTesting: true, remote: {}, browser: null, specs: [], proxyUrl: 'http://localhost:37763', browserUrl: 'http://localhost:8080/__/', reporterUrl: 'http://localhost:8080/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:37763', state: {} } +372ms
cypress:network:cors Parsed URL { port: '37763', tld: 'localhost', domain: '' } +372ms
cypress:server:remote-states getting remote state: { auth: undefined, origin: 'http://localhost:37763', strategy: 'file', fileServer: 'http://localhost', domainName: 'localhost', props: null } for: http://localhost:37763 +372ms
cypress:server:routes proxying static assets /__/assets/chrome.1b5cb774.svg, params[0] chrome.1b5cb774.svg +519ms
cypress:server:project socket:connected +526ms
cypress:server:run got socket connection { id: 'qcjyc5ci34' } +963ms
cypress:server:run socket connected { socketId: 'qcjyc5ci34' } +0ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/main.js +3s
cypress:server:server-base Got CONNECT request from 127.0.0.1:8080 +566ms
cypress:https-proxy Writing browserSocket connection headers { url: '127.0.0.1:8080', headLength: 0, headers: { host: '127.0.0.1:8080', 'proxy-connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/103.0.5060.53 Safari/537.36' } } +568ms
cypress:https-proxy Got first head bytes { url: '127.0.0.1:8080', head: 'GET /ws HTTP/1.1\r\nHost: 127.0.0.1:8080\r\nConnection: Upgrade\r\nPra' } +0ms
cypress:https-proxy Making intercepted connection to 37763 +1ms
cypress:network:connect successfully connected { opts: { port: 37763, host: 'localhost', getDelayMsForRetry: [Function: getDelayForRetry] }, iteration: 0 } +569ms
cypress:https-proxy received upstreamSocket callback for request { port: 37763, hostname: 'localhost', err: undefined } +0ms
cypress:server:util:socket_allowed allowing socket { localPort: 42184 } +567ms
cypress:server:server-base Got UPGRADE request from /ws +2ms
cypress:network:cors Parsed URL { port: '8080', tld: '127.0.0.1', domain: '' } +559ms
cypress:network:agent addRequest called { isHttps: false, href: 'http://127.0.0.1:8080/ws' } +4s
cypress:network:agent got family { family: 4, href: 'http://127.0.0.1:8080/ws' } +0ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/static/js/vendors-node_modules_react_index_js.chunk.js +44ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/static/js/vendors-node_modules_cypress_react_dist_cypress-react_esm-bundler_js.chunk.js +1ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/static/js/cypress_support_component_js.chunk.js +1ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/static/js/vendors-node_modules_css-loader_dist_runtime_api_js-node_modules_css-loader_dist_runtime_sour-3ed60d.chunk.js +70ms
cypress:server:routes-ct proxying to /__cypress/src, originalUrl /__cypress/src/static/js/spec-1.chunk.js +2ms
cypress:server:project received runnables { id: 'r1', title: '', root: true, type: 'suite', file: 'src/App_91.cy.jsx', retries: -1, _slow: 250, hooks: [], tests: [ { id: 'r2', title: '[App_91.cy.jsx]: renders learn react link', body: '() => {\n' + ' cy.mount( /*#__PURE__*/(0,react_jsx_dev_runtime__WEBPACK_IMPORTED_MODULE_1__.jsxDEV)(_App__WEBPACK_IMPORTED_MODULE_0__["default"], {}, void 0, false, {\n' + ' fileName: _jsxFileName,\n' + ' lineNumber: 4,\n' + ' columnNumber: 12\n' + ' }, undefined));\n' + ' cy.get("a").contains("React");\n' + '\n' + ' for (let i = 1; i < 10; i++) {\n' + ' cy.get("button").click();\n' + ' cy.get("p").contains(i);\n' + ' }\n' + '}', type: 'test', file: null, invocationDetails: [Object], currentRetry: 0, retries: -1, _slow: 250, hooks: [Array], _testConfig: [Object], _titlePath: [Array] } ], suites: [], runtimeConfig: {} } +151ms
cypress:server:reporter trying to load reporter: spec +3s
cypress:server:reporter spec is Mocha reporter +0ms
cypress:server:project onMocha start +13ms
Summary
Ok, I looked into it. It turns out creating a new tab (and associated processes) does impact the performance. Here's a branch with the 9.7.0 behavior, which is one tab -> run all specs in there, as opposed to the current implementation which is to create a new tab per spec.
- Branch: lmiller/22353-ct-single-session.
- Diff.
- Test repo I created with 100 specs.
There's a build of this branch you can install for linux x64, get it here. There's likely some bugs, but npx cypress run --component seems to work fine - if you are running linux, you could give it a try. I'll get a more reliable build up for windows/mac soon.
Note: these numbers are from 1 time runs - not the most scientific. To be more accurate, I'd run them all many times and take the average, but these are good enough to validate that opening a new tab is the primary reason we are slower.
Toy repo
| Metric | 9.7.0 | 10.3.0 | 10.3.0 (one tab) |
|---|---|---|---|
| Run time | 175s | 226s | 146s |
| Average ms per spec | 899ms | 984ms | 860ms |
For the run time: (+18% Cypress 9.7.0, +37% Cypress 10.3.0)
These numbers are a bit exaggerated, since I'm running 1 test per spec, which is not common. Normally, a single spec file will have 10s or 100s of tests. Either way, this experiment implies a ~300ms cost for creating a new tab, which is significant.
picasso repo https://github.com/toptal/picasso (run locally)
| 9.7.0 | 10.3.0 | 10.3.0 (one tab) | % improvement |
|---|---|---|---|
| ? | 138s | 78s | 43% |
It's 43% faster! I guess this mostly accounts for the 2x slowdown described.
Cypress internal component tests (local, no parallel, no video)
| package | 10.3.0 | 10.3.0 (one tab) | # specs | % improvement |
|---|---|---|---|---|
| packages/frontend-shared | 150s | 106s | 41 | 29% |
| packages/app | 326s | 232s | 75 | 28% |
| packages/launchpad | 102s | 73s | 27 | 28% |
Run locally. I turned off video, which makes everything slower in a linear fashion, so it shouldn't impact the relative differences. 30% is a solid improvement.
Discussion
I think it's death by many a thousand paper cuts - just opening a new tab is not a single performance killer, but it looks like we do some cleanup between specs, and reset a bunch of things. I need to go deeper, but I suspect at least some of these things are absolutely necessary to guarantee spec isolation and determinism in E2E, which is why E2E has done the "new browser per spec" things since forever. I'll look more into this, and if it's necessary for component testing.
As shown on my toy repo with 100 of the space spec, as as predicted, the actual test execution time is identical, since your production code and specs hasn't changed changed.
Next Steps
- build pre-release binary, share download link and solicit feedback - although it's faster in my example repo and picasso, is it also faster for everyone else?
- investigate implications of same-tab vs new-tab (for component testing only)
- consider implications of slightly different behavior for E2E (new tab per spec) and CT (same tab per spec) w.r.t performance
Thanks for the patience and assistance here everyone! I think we are on the right track. I'll be focused on this until we have the performance issue solved.
If anyone is following this and wants to give me some feedback, that'd be great. For the next 48 hours you can grab a pre-release for linux and mac here: https://github.com/cypress-io/cypress/commit/6e4d8cd1b3c1c4f3a32e002ea03ca77cb49bb191 (scroll down to the comments). It's possible E2E is a bit bugged in this build; I only tested it on component testing thoroughly, since the main goal is to fix the performance improvements.
The change from one tab -> one tab per spec for component testing was to improve spec isolation, and make CT and E2E behave in a similar fashion. It actually resulted in a speed UP for E2E, which previously created a new browser for each spec. I think, ideally, we want the best of both worlds; faster component tests and better spec isolation, along with E2E consistency.
I'll keep looking into this and post updates here.
@lmiller1990 I have tested it out in Picasso and it works perfectly! 🚀 🎉
I really enjoyed following this issue and checking how you are handling it, especially since you share your thought process when debugging!
Looking forward to this being released.
Thank you for your work! Keep rocking! 💪 🙇
I tested 4 versions of Cypress with our smallest project that only has a handful of complex components that require many assertions to fully test, so most of them are fairly small test cases
From my tests, it appears either 10.2.0 or 10.3.0 resolved most of the performance slowdown found in 10.0.0 and 10.1.0 - I presume its from proper ARM support for Linux/MacOS in 10.2.0.
There did not appear to be any significant reduction in test setup/cleanup duration from component tests being performed in one tab like with the Picasso and internal Cypress tests over consecutive runs of 10.3.0 and the Circle CI build (this macOS ARM build) on my machine with this small Vue project (both versions are already faster than Cypress 9 though!)
Tests
These were run twice each with the times of the 2nd run recorded (so if anything is happening when "first time using x" appears doesn't impact times)
All tests were with video recording disabled and using Cypress' Vue & Vite configuration
Note: Skipped tests were disabled as they failed with run but pass with open in 10.0.0 (I haven't found why yet)
Individual tests per component spec file
| Component | 9.7.0 | 10.0.0 | 10.3.0 | Circle CI |
|---|---|---|---|---|
| callToAction | 573ms | 686ms | 151ms | 115ms |
| headerNavigation | 141ms | 1075ms | 183ms | 116ms |
| howItWorks | 157ms | 732ms | 138ms | 125ms |
| howItWorksCard | 155ms | 431ms | 121ms | 150ms |
| tButton | 830ms, 223ms | Skipped | Skipped | Skipped |
| tInput | 395ms, 291ms | 1001ms, 817ms | 326ms, 216ms | 296ms, 217ms |
| tDots | 66ms, 64ms | 474ms, 156ms | 80ms, 31ms | 84ms, 33ms |
| tFooter | 198ms | 825ms | 114ms | 117ms |
| tLanding | 136ms | 783ms | 128ms | 168ms |
| contactSelection | 3018ms | 3874ms | 1140ms | 1222ms |
| demoTitle | 114ms, 72ms | 275ms, 103ms | 75ms, 52ms | 48ms, 25ms |
| labelledInput | 331ms, 414ms | 1029ms, 636ms | 333s, 310ms | 328ms, 315ms |
| placesInput | 3482ms | 3701ms | 1818ms | 1782ms |
| stepNavigation | 73ms, 79ms, 37ms, 53ms | 239ms, 86ms, 69ms, 138ms | 65ms, 17ms, 16ms, 20ms | 36ms, 23ms, 22ms, 19ms |
| submitData | 1223ms | 2282ms | 1104ms | 1209ms |
| home | 6412ms | 10,262ms | 5048ms | 5160ms |
Total duration of each component spec file according to Cypress
| Component | 9.7.0 Spec Total |
10.0.0 Spec Total |
10.3.0 Spec Total |
Circle CI Spec Total |
|---|---|---|---|---|
| callToAction | 644ms | 1s | 211ms | 127ms |
| headerNavigation | 195ms | 1s | 205ms | 215ms |
| howItWorks | 170ms | 860ms | 145ms | 137ms |
| howItWorksCard | 316ms | 561ms | 134ms | 168ms |
| tButton | 1s | Skipped | Skipped | Skipped |
| tInput | 706ms | 2s | 559ms | 586ms |
| tDots | 164ms | 876ms | 133ms | 138ms |
| tFooter | 210ms | 933ms | 124ms | 125ms |
| tLanding | 149ms | 1s | 139ms | 174ms |
| contactSelection | 3s | 4s | 1s | 1s |
| demoTitle | 201ms | 564ms | 198ms | 174ms |
| labelledInput | 776ms | 2s | 662ms | 748ms |
| placesInput | 4s | 4s | 2s | 2s |
| stepNavigation | 278ms | 684ms | 139ms | 129ms |
| submitData | 1s | 3s | 1s | 1s |
| home | 6s | 10s | 5s | 5s |
Total test duration (Yarn's command execution timer)
| 9.7.0 | 10.0.0 Total |
10.3.0 Total |
Circle CI Total |
|---|---|---|---|
| 51s (50s without tButton) |
140s | 40s | 45s |
@TomasSlama cool, thanks for the support and help testing!!
@wopian can you confirm your table info a bit? When you say 10.3.0 you are referring to my build (not the live 10.3.0 on npm?)
Those are some huge improvements, good to see!
Just to clarify a bit more here - unfortunately it's not as simple as "just ship these changes". Opening a new tab clears a lot of things and better isolates the specs, which is a feature of any good test runner. The original goal of this change was to be more in line with E2E - you can image it'd be confusing if CT and E2E diverge. I will find out exactly what we do better each spec and find out how to do it in a faster way than what we do now.
My next step is for me to dig deeper and isolate more closely what's really killing performance - I think it's more than just "creating a new tab", but also things like teardown/setup/rendering the runner UI.
The end goal is to improve performance for both CT and E2E. The spec execution flow is at the core of Cypress. It needs a bit more investigation, but we are on the right track with this thread. We take this kind of regression (performance, and more importantly determinism and reliability) very seriously. Just to clarify, this is a performance regression for Component Testing - this change made E2E faster, since it use to spin up a new browser for each spec, now it just does a new tab.
I am out next week, but after that I'm going to look into a more detailed technical breakdown, then we can actually start moving forward with some updates. The Component Testing "alpha" label was there for reason, and I'd like to move out of "beta" sooner than than later, but that means exercising great caution around mission critical things like performance and spec isolation.
I will continue posting my updates here, I think it's fine to take a little time as long as we are clear with communication. This is still my main focus and will be until it's we are back to a comparable, if not faster, state than the 9.x Component Testing alpha.
When you say 10.3.0 you are referring to my build (not the live 10.3.0 on npm?)
10.3.0 on npm
One listed as Circle CI is the 10.3.0 from your build (used the ARM build linked in my comment). I can double check if it did correctly install and use your build or if I accidentally used an outdated build tomorrow
@wopian can you confirm your table info a bit? When you say 10.3.0 you are referring to my build (not the live 10.3.0 on npm?)
It appears the installation failed to use the .tgz when I ran the tests for it last week.
Confirming it was running your branch with the macOS ARM build, the same tests completed in 20-22s - a >50% improvement on 10.3.0 (live) and an >80% improvement on 10.0.0! Awesome 🎉
The pre-release builds are only available for a few days. Maybe it's expired. I can make a new one if needed. At this point, it's pretty clear what's slowing us down, though, I don't see a ton of value in getting more testing in at this point.
Making these changes isn't super easy, since there's a lot of complexity here, but I'm bringing this up internally as a priority. I'll post updates here w.r.t what we are doing, when it can expect to land, etc.
Re: ARM builds - yes they absolutely fly, although I am still on Intel, I'd like to make Intel fly, too.
Sorry for the delay on this one, folks. Here's my update. After more research, there are two main things killing us between specs.
- Time to inject & parse
cypress_runner.js: ~700ms - Everything else (render, connect to websocket, bootstrap Cypress AUT IFrame, start executing spec: ~800-900ms
Actual rendering of the UI looks like it's not a major bottleneck: ~150ms. There's also little things that cost us each spec, like establishing a websocket connection (~40ms), rendering command log (~100ms) etc.
Edit run the benchmarks regarding cypress_runner.js parsing in isolation: https://github.com/lmiller1990/cypress-runner-inject-benchmarks
Most of these happen in parallel, but they all need to finish so we can start executing the spec. These are all one-time things (per tab). So, over many specs, it adds up.
The primarily culprit, I think, is Cypress serves and injects a cypress_runner.js (see: driver) file that is really huge (several MB, around 10k LOC, including things like jquery, lodash etc). We spend about 700ms parsing this entire thing (there's nearly no network latency, it's served locally).
By re-opening the browser each spec, we incur a 700ms penalty per spec. Allegedly, browsers should be able to cache and not reparse, but this isn't my observation. Note parsing cypress_runner.js is not a blocking operation.
We inject the bundle here. We could add defer = true to potentially get some optimizations.
All of these things make for a total of ~1000ms to ~1500ms per spec. Let's go with 1500ms to be pessimistic. Looking at the numbers I posted before:
| package | 10.3.0 | 10.3.0 (one tab) | difference | # specs | ms improvement / spec |
|---|---|---|---|---|---|
| packages/frontend-shared | 150s | 106s | 44s | 41 | 41*1.5s = 60s |
| packages/app | 326s | 232s | 75 | 94s | 75 * 1.5s = 112s |
| packages/launchpad | 102s | 73s | 29s | 27 | 27 * 1.5s = 40s |
This pretty much lines up with my observations.
So, if we want to see speed like CT in 9.x, we'd need to somehow:
- eliminate need to parse
cypress_runner.js. I don't know how to do this, or if it's even possible. - find a way to pre-render or persist things across specs (that wouldn't impact spec isolation). There isn't an obvious bottleneck to start with, though, so this would likely be a slow, incremental task.
Neither of these has obvious solutions, from what I can see. The change (tab per spec) was made to make CT more in line with E2E, and avoid specs leaking. The motivation for this behavior is shared in great detail when it was added, back in the 3.x days. CT probably should have been like this from the start.
For these reasons, I do not see a good way to replicate the speed of CT in 9.x without incurring the downsides (worse spec isolation). Hacking back in the "one tab for all specs" approach like I initially did to validate the problem was a good exercise to understand the cause of the slowdown, but doesn't seem like a good solution for a test runner, where reliability is the #1 priority.
Open to suggestions here - anyone have any thoughts/feedback on this?
@lmiller1990 - One idea that comes to mind - and this is not well thought out, it is just off the cuff - is what would happen if we opened the new tab and loaded in cypress_runner.js before closing the old tab? I wonder if this would help with caching and not re-parsing, if the first tab is still open (and therefore the browser hasn't had time to flush anything from memory.
The stack overflow link suggests that the cache is not cleared when the page is reloaded - but my understanding is that we close the tab and re-open it, rather than reloading the page in-place? Perhaps that would be a change worth exploring as well. I've actually wondered if this behavior is also causing an issue we see in Electron, where it crashes after too many specs in run mode.
@BlueWinds cool idea - I'll definitely look into this. I'm fairly sure we:
- open tab for spec 1
- do the things
- close tab
- open tab for spec 2
- repeat
I created a minimum repository for quick testing: https://github.com/lmiller1990/cypress-runner-inject-benchmarks. Average time on GH Actions is ~600ms - this is just injecting the script, not doing any rendering or loading anything else.
It does it 100 times and takes average.
I can try your idea quickly with this setup - thanks! I'm glad this is becoming "make Cypress faster" rather than "make CT faster". This is one reason CT changed to be in line with E2E - so any work we done has a greater impact across the entire Cypress experience.
I wonder if it would be even better to reuse the existing tab, rather than closing -> opening in the first place. Could be done by either navigateing to new URL -> location.reload(), or by navigating to about:blank -> navigate to new spec.
@BlueWinds Another good idea... I don't think the "delay closing tab" strategy helps: https://github.com/lmiller1990/cypress-runner-inject-benchmarks/commit/4509679cb19fa36b203275d17ccde9c105d78303
Bench: https://github.com/lmiller1990/cypress-runner-inject-benchmarks/runs/7420972670?check_suite_focus=true#step:5:110 (578ms average).
I'll also explore the re-navigate strategy.
Another question would be what cross-test contamination are we actually clearing by using a new tab? If it's just window variables and mutation, I think a lot of users would be happy to be careful and aware of some potential cross-spec contamination as a trade-off to run specs way faster. I'm guessing there's more things to consider, like cookies, sessions, etc (haven't looked into this area of Cypress that much yet).
It's a bigger discussion, but it comes back to the whole Run All Specs conversation. Things like snapshots consuming lots of memory are not a concern for a run-mode-only improvement, like the one we are exploring here. Although Run All Specs had (has) problems, it worked beautifully (or at least, good enough) for some users. I wonder if these performance improvements we are talking about here will need to be part of a bigger project relating to Run All Specs.
One more thought - does our server indicate cypress_runner.js is cacheable? I'm not sure it would help, but it's another thing to examine while you're in there.
Good idea. I will look at the headers our server sends.
I just noticed having devtools open majorly impacts some measurements. I'm going to need to re-verify some of these ones.
Interesting. New tab seems to be a no-go, but location.reload() (or some other single tab strategy) looks positive. I will link the repo and numbers in a moment.
The second benchmark doesn't reload, so it's not really an option. The third, location.reload() appears about 6x faster. Link.
Since we need to change the href, too, we might do document.location = .....?spec=newSpec for a similar behavior.
Basic summary of above:
Some specific exploration to avoid re-parsing cypress_runner.js, which takes a non trivial amount of time, was was conducted. Note that this was done in isolation to the runner - a minimal web page that does nothing but fetch and parse cypress_runner.js over and over was used for the benchmarks below. You can see the results here and the basic code is here. The README has instructions on running the benchmarks yourself.
-
Some benchmarks. We can potentially replace the new tab strategy with a single tab +
document.location.reloadordocument.location.href = <next_spec_url>in the future. This might be faster than the current approach, since the browser does not re-parsecypress_runner.jsin the same tab, but it does re-parse it when opening a new tab.
Here are some basic numbers:
| Implementation | Run 1-3 | Run 98-100 | Mean Time to Parse |
|---|---|---|---|
| New tab | 617ms, 557ms, 407ms, | 423ms, 412ms, 409ms | 423ms |
| Single tab, no reload | 553ms, 354ms, 88ms | 58ms, 59ms, 55ms | 67ms |
Single tab, document.reload() |
537ms, 356ms, 83ms | 93ms, 88ms, 91ms | 106ms |
Hello all, it looks like doing a significant change to how we run specs is going to be a more longer term project that ties in with the "Run All Specs" feature discussion. This project touches similar issues (how to run many specs quickly).
For a short term fix, it's been suggested we have an opt in flag to replicate the functionality of CT (Component Testing) in 9.x, which is to run all specs in a single tab, sequentially. This would come with the caveats that are listed in the first post of Run All Specs. This would be for CT only - E2E never worked like this.
As for how to opt into this, It would likely be listed as experimental, so you'd opt into it - so something like experimentalComponentSingleSession. This will also allow for future experimentation, like using location.reload or an alternative strategy, such as those listed above.
I will post more updates as they come. Thanks for the patience everyone!
Just came across this issue because I'm having problems with E2E tests being painfully slow. I don't have official measurements, but the tests are pretty much unusable for local dev. I'm planning to open an issue, but wanted to be sure that it's not being handled in this issue before I do. (We're using Vite which I thought had something to do with it, but Vite doesn't appear to be the issue as loading things locally - outside of Cypress - is snappy). I'm assuming I should open a new issue, right? This issue is just for slow component tests?
Yes this issue is just for component tests.
The Vite Dev Server integration is only used for Component Tests, though - E2E does not use a dev server, so I don't think Vite has anything to do with your problem.
Definitely worth opening a new issue, try to include:
- when it got slow (Cypress 9->10)
-
cypressfiles (likecypress.json,cypress/plugins/index.js - OS
Got it, thanks Lachlan! For anyone else who may be running into slow E2E tests, I opened #22968.
We are facing a similar problem with component testing.
In our case we have a monorepo with a structure like:
custom-ui-framework. app-a (depends on custom-ui-framework). app-b (depends on custom-ui-framework). We are using:
cypress 10.3.1 + @cypress/vue2 1.0.1 vue 2.6.14. vite 2.9.14. We can notice that when running component tests on any package, all the components from the custom-ui-framework are being fetched, no matter if they are being used or not by the tests.
I attach the cypress and vite config just in case it is useful to you.
cypress.config.ts
export default defineConfig({
component: {
devServer: {
framework: 'vue',
bundler: 'vite',
},
specPattern: 'src/**/*ct.spec.{js,jsx,ts,tsx}',
supportFile: 'cypress/support/component.ts',
video: false,
},
});
vite.config.ts
/// <reference types="vitest" />
import { defineConfig } from 'vite';
import { createVuePlugin } from 'vite-plugin-vue2';
import path from 'path';
import ViteFonts from 'vite-plugin-fonts';
import { visualizer } from 'rollup-plugin-visualizer';
// https://vitejs.dev/config/
export default defineConfig({
build: {
rollupOptions: {
plugins: [
visualizer({
open: false,
gzipSize: true,
}),
],
},
},
plugins: [
createVuePlugin(),
ViteFonts({
google: {
families: [
{
name: 'Inter',
styles: 'wght@100;200;300;400;500;600;700;800;900',
},
],
},
}),
],
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
},
},
test: {
environment: 'jsdom',
include: ['**/*.unit.spec.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
setupFiles: ['./__tests__/setup.js'],
coverage: {
reporter: ['text', 'json', 'html'],
include: ['src/composables', 'src/stores', 'src/utils'],
all: true,
},
},
server: {
fs: {
allow: ['..'],
},
},
});
@mverdaguer what does your support/component.ts look like? Vite should only fetch dependencies that are imported in some fashion. We are just a thin layer on top of Vite - although it's definitely possible we've got a bug. I don't see any obvious problem in your configuration.
If you do supportFile: false, does that stop the fetching? Just one idea to help debug. I suspect your issue isn't directly related - are you experience this only after going from v9 -> v10? If so, it could be - if not, I'd recommend a new issue titled something like "Vite loading unrelated dependencies", since it sounds like that's closer to what you are experiencing.