cypress

cypress copied to clipboard

Current behavior

I have an existing cypress test suite which runs fine in Chrome and Electron, i have experimented with webkit , when the cypress open command runs and selected with Webkit, it resulting to blank page.

playwright-webkit added with command : yarn add --dev playwright-webkit

Engine: node : v18.0.0 yarn : 1.22.18

Specification : "cypress": "~10.11.0" "playwright-webkit": "~1.28.0"

OS: macOS : v12.6

Cypress Open

Cypress Run

Cypress started in fd-sb-aw mode [0] [0] ==================================================================================================== [0] [0] (Run Starting) [0] [0] ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ [0] │ Cypress: 10.11.0 │ [0] │ Browser: WebKit 16 (headless) │ [0] │ Node Version: v18.0.0 (/opt/homebrew/Cellar/node/18.0.0/bin/node) │ [0] │ Specs: 71 found (.........) │ [0] │ Searched: /..//*.js │ [0] │ Experiments: experimentalInteractiveRunEvents=true,experimentalWebKitSupport=true │ [0] └────────────────────────────────────────────────────────────────────────────────────────────────┘ [0] [0] [0] ──────────────────────────────────────────────────────────────────────────────────────────────────── [0]
[0] Running: fd-sb-aw/account/sso_recovery.spec.js (1 of 71) [0] [0] Your project has set the configuration option: chromeWebSecurity to false. [0] [0] This option will not have an effect in Webkit. Tests that rely on web security being disabled will not run as expected. [0] [0] Timed out waiting for the browser to connect. Retrying...

Desired behavior

Like Chrome and Electron, it should load the identified spec in webkit

Test code to reproduce

Engine: node : v18.0.0 yarn : 1.22.18

Specification : "cypress": "~10.11.0" "playwright-webkit": "~1.28.0"

OS: macOS : v12.6

Cypress Version

10.11.0

Node version

v18.0.0

Operating System

12.6

Debug Logs

No response

Other

No response

Hi @manoj-fd, to try and set a baseline, i put together a super simple repo with webkit setup: https://github.com/mjhenkes/simple-webkit-example

With that repo, the webkit browser opened up for me and i was able to run the generated test. Could you pull down that repo and see if webkit opens for you?

Hi @mjhenkes , I have already tried similar sample project set up and works fine, however when i tried with my existing project, it shows blank page and below is the error in debug mode

[0]   cypress:server:browsers getBrowserLauncher { browser: { name: 'webkit', channel: 'stable', family: 'webkit', displayName: 'WebKit', version: '16.4', path: '/Users/manoj.gangadharan/Library/Caches/ms-playwright/webkit-1735/pw_run.sh', majorVersion: '16', warning: 'WebKit support is currently experimental. Some functions may not work as expected.', isHeadless: true, isHeaded: false } } +1s
[0]   cypress:server:cypress exiting with err Error: connectToNewSpec called without wkAutomation
[0]     at Object.p (<embedded>:2760:19005)
[0]     at Object.connectToNewSpec (<embedded>:2760:24813)
[0]     at async v.relaunchBrowser (<embedded>:2812:346537) +3m
[0] connectToNewSpec called without wkAutomation
[0] Error: connectToNewSpec called without wkAutomation
[0]     at Object.p (<embedded>:2760:19005)
[0]     at Object.connectToNewSpec (<embedded>:2760:24813)
[0]     at async v.relaunchBrowser (<embedded>:2812:346537)

We are using couple of features in our project along with cypress.

• Applitools dependencies installed - @applitools/eyes-cypress": "3.27.1",
• Cypress axe installed -

  • "cypress-axe": "~1.0.0",
    

  • "cypress-file-upload": "~5.0.8",

  •  "cypress-iframe": "~1.0.1",
    

Also we are currently using multiple cypressconfig.ts file, [cypress-buildkite.config, cypress-e2e.config, cypress-local.config]

@manoj-fd, could you attach your config files? If the simple example works there must be something in either a dependency or your config files that is causing the issue.

Hi @mjhenkes ,Please refer the cypress.config.js and index.js config mentioned below , if you can use that with your example project, the issue can be replicated.

As a summary, whenever we try to change the Baseurl to a different local host , webkit is not launching the spec grid and its blank, but the same works fine with chrome and electron.

Cypress.config.js

module.exports = {
  e2e: {
    setupNodeEvents(on, config) {
      return require('./index.js')(on, config)
   },
    experimentalWebKitSupport: true,
  },
}

Index.js

module.exports = (on, config) => {
    
    Object.assign(config, {
        baseUrl: 'https://localhost:3001',
    })

    return config
}

I can confirm that this causes webkit to open blank.

Though i'm curious why you don't just set baseurl config directly in the cypress config

module.exports = {
  e2e: {
    baseUrl: 'http://localhost:3001',
    setupNodeEvents(on, config) { })

    return config
    },
    experimentalWebKitSupport: true,
  },
}

Thanks for providing the reproduction!

Hi @mjhenkes , We are using multiple cypress config files based on test environment, ideally the logic to return the base url is handled in index.js and cypress config files driven based on that.

Hi @manoj-fd, I started investigating this a bit and had a quick question off the jump. Looking at your example index.js:

module.exports = (on, config) => {
    
    Object.assign(config, {
        baseUrl: 'https://localhost:3001',
    })

    return config
}

Are you actually serving that route over https? I ask because I was able reproduce the blank webkit based on the prior posts, but I saw an error in the WebKit console that pointed out that my local server wasn't actually serving over https:

Changing the baseUrl to use http corrected this, as that is how my local server was actually serving it.

I also ask because you mention that your config works in Chrome and Electron. Those browsers (as well as Firefox) are more graceful when it comes to the protocol mismatch and will show that the connection is not secured in the url bar rather than fail, which could explain the differences in behavior you're seeing:

So I'm just curious if this lines up with what you're seeing locally, and if using http rather than https corrects your immediate issues launching WebKit. If so, there's definitely work to be done with Cypress to actually surface these errors in the UI. If not, we'll have to do a bit more digging.

Hi @tbiethman , Thanks for investigating the issue.

Yes, changing the baseUrl to use http in the Sample project, launches webkit with spec details.

However in the actual project, the application is spin up in https protocol, changing https to http in the actual project baseurl resulting to below error

@manoj-fd Thank you for confirming. That means I'm barking up the wrong tree. I don't think the error in the sample project is the same that you're seeing in your project, though it has the same functional result (webkit launching blank).

Can you confirm with your original setup:

• Whether the WebKit browser console displays any errors/warnings? You can open the console with Cmd+Option+I after it launches.
• Whether there are any other errors present in the Cypress debug logs? I see the Error: connectToNewSpec called without wkAutomation error you posted, but I'm curious if there are any errors printed prior to that. A full set of cypress:* logs from Cypress launch to the connectToNewSpec error would be most helpful.

Hi @tbiethman , I have attached the debug logs with cypress:*, please let me know if its helpful logs.txt

Thanks @manoj-fd , that log helps explain why you're seeing the connectToNewSpec called without wkAutomation error. We launch the browser instance for the first spec and then subsequently reuse it for all following specs. It looks as though the browser is failing to launch for your first spec, which is causing the subsequent spec runs to throw that exception when started.

To get you testing, we need to track down why the initial launch is failing.

• The logs you posted start right when the first spec has failed; can you post the prior logs prior to that? Access to the whole set of logs, from Cypress launch until the exception is thrown, would be ideal.
• Were you able to check if the WebKit browser itself renders any errors/warnings in its console when it's launched and blank?

Hi @tbiethman , I have attached the Webkit console logs and Cypress debug logs for further reference. cypress-run-logs.txt webkit-console-log.txt

This error is shown in the WebKit console log:

page.goto: The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 310.)

Can you verify whether or not your local endpoint (https://localhost:3001) can be loaded successfully in playwright-webkit outside of Cypress (you can launch with npx playwright-webkit wk), or even in Safari?

Also, it does seem as though the cypress-run-logs.txt file is still a subset of the full Cypress logs. I'm not seeing evidence there of our internal proxy server starting appropriately, or failing to start, which could also be the problem here. If you could provide the full set of logs starting from your DEBUG=cypress:* npx cypress open (or equivalent) command, it would be a big help to rule out some error cases.

@tbiethman I will share the complete logs shortly

Please refer the below screenshot where local endpoint tried to be accessed in Playwright browser and Safari.

Playwright Browser -

Console error - Failed to load resource: The certificate for this server is invalid. You might be connectina to a server that is pretending to be "localhost" which could put vour confidential information at risk httos://localhost:3001/

Safari Browser -

Upon initial Launch -

After Clicking Proceed, it successfully navigating to login page

Hmm, the playwright-webkit error is interesting. Cypress sets the --ignore-https-errors option on the instance it launches. I'm curious if launching playwright-webkit with https errors ignored (npx playwright-webkit wk --ignore-https-errors) allows your page to load successfully, or if it continues to fail?

If it loads with https errors ignored when launched outside of Cypress, it's a pretty good indication that Cypress' internals are causing your issues. But if it doesn't work, or if it throws the kCFErrorDomainCFNetwork error 310 we've seen previously, it could indicate that there's an incompatibility between your local server and playwright-webkit itself.

Hi @tbiethman , the app getting launched successfully with (npx playwright-webkit wk --ignore-https-errors), it navigates to https://localhost:3001/login.

There is a slight change in the behaviour when i run with DEBUG=cypress:* npx cypress open,

1. After clicking webkit in cypress, it opens blank page and then navigates to https://localhost:3001/__/ for few secs(approx 3 secs)
2. And then the webkit shows the login page of the application in https://localhost:3001/login,

But the expectation is, it has to open the spec dashboard to display the specs. (that's the behaviour with other browsers)

Please refer the complete logs. full_logs.txt.zip

Chrome Behaviour -

Webkit

For me, the issue was in playwright-webkit v1.29.0. Downgrading it to v1.28.0 fixed the issue @manoj-fd

edit: make sure you removed the node_modules and package-lock.json

Thanks @vladlearns , I have been trying this with playwright-webkit V1.28.0 since beginning and the issue is still reproducible for me.

@vladlearns have retried with WebKit version 1.26.0 and similar behaviour as mentioned in https://github.com/cypress-io/cypress/issues/24767#issuecomment-1357769447

I've been looking into this and it seems that webkit has issues with the hostname being localhost that may be causing requests not to go through Cypress's proxy. I don't think there's anything we can do within Cypress today to work around it, so I've filed an issue with playwright in case something can be done there.

In the meantime, you might be able to work around the issue by using an alternative string for https://localhost, such as the IP address of your computer or by using https://get.localhost.direct/.

I've been looking into this and it seems that webkit has issues with the hostname being localhost that may be causing requests not to go through Cypress's proxy. I don't think there's anything we can do within Cypress today to work around it, so I've filed an issue with playwright in case something can be done there.

In the meantime, you might be able to work around the issue by using an alternative string for https://localhost, such as the IP address of your computer or by using https://get.localhost.direct/.

Using the IP directly also did not work for me, serving WebKit without the spec page, I believe this is more about http vs https

For me only deleting e2e baseURL from cypress.config.js file helps to fix this issue, but it is not valid way for me...