cypress icon indicating copy to clipboard operation
cypress copied to clipboard

We encountered an unexpected error communicating with our servers. RequestError: Error: self signed certificate in certificate chain, Error for connection with dashboard api.

Open Oppon2 opened this issue 1 year ago • 13 comments

Current behavior

We run cypress in gitlab and get connection error to dashboard api showing certificate signature failure. Everything works fine in 10.11.0. With the version 12.12.0 we started to see the below error

We encountered an unexpected error communicating with our servers. RequestError: Error: self signed certificate in certificate chain

We set the strict-ssl to false but still facing this issue

Desired behavior

In 10.11.0 everything is fine

Test code to reproduce

$env:NODE_TLS_REJECT_UNAUTHORIZED=0 npm config set strict-ssl=false

Cypress Version

12.12.0

Node version

v18.16.0

Operating System

Windows11

Debug Logs

No response

Other

No response

Oppon2 avatar May 18 '23 15:05 Oppon2

Hi @Oppon2 thank you for submitting this issue. I am sorry to hear you are having trouble with Cypress Cloud. If you are a paying user of the cloud please head over to our Cypress Support Portal to receive our fastest support.

nagash77 avatar May 18 '23 18:05 nagash77

@Oppon2 Could you run Cypress in debug mode mode and print the entire set of logs here?

warrensplayer avatar Jun 19 '23 20:06 warrensplayer

@Oppon2 Cypress version 11 introduce strict certificate validation as referenced in the changelog.

Communication with the Cypress Dashboard will now verify CAs and reject any unauthorized calls. If you use a self-signed CA you will need to set npm_config_ca, npm_config_cafile, or NODE_EXTRA_CA_CERTS.

Have you tried making the changes suggested there?

mike-plummer avatar Jun 22 '23 18:06 mike-plummer

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information to provide concerning the original issue and we can reopen.

mike-plummer avatar Jun 29 '23 14:06 mike-plummer

I was running into this too, so I thought setting NODE_EXTRA_CA_CERTS should solve the problem, it does not, now I am getting this:

We encountered an unexpected error communicating with our servers.

DecryptionError: General JWE must be an object

A google search for "General JWE must be an object" does not yield any results at all, so I'm 100% lost here. Any help is greatly appreciated @mike-plummer

rryter avatar Jul 03 '23 19:07 rryter

Some log output:

cypress:server:cloud:api request to url: POST https://api.cypress.io/preflight with params: {"body":{"apiUrl":"https://api.cypress.io/","dependencies":{},"errors":[],"projectId":"ahkfhc","ciBuildId":null,"testingType":"e2e","parallel":null},"headers":{"x-route-version":"1","x-cypress-request-attempt":0,"x-os-name":"linux","x-cypress-version":"12.15.0"}} and token: undefined +1m
  cypress:network:agent addRequest called { isHttps: true, href: 'https://api.cypress.io/preflight' } +1m
  cypress:network:agent got family { family: undefined, href: 'https://api.cypress.io/preflight' } +0ms
  cypress:network:agent Creating proxied socket for https://api.cypress.io/preflight through http://www-proxy.visana.ch:8080/ +1ms
  cypress:network:connect successfully connected { opts: { port: 8080, host: 'www-proxy.visana.ch', useTls: false, getDelayMsForRetry: [Function (anonymous)] }, iteration: 0 } +0ms
  cypress:network:agent Proxy socket for https://api.cypress.io/preflight established +19ms
  cypress:server:record failed creating run with status { name: 'DecryptionError', message: 'General JWE must be an object', stack: 'DecryptionError: General JWE must be an object\n' + '    at e.transform (<embedded>:4353:47610)\n' + '    at process.processTicksAndRejections (node:internal/process/task_queues:96:5)' } +0ms
  cypress:server:cypress exiting with err Error
    at S (<embedded>:4629:437270)
    at <embedded>:4629:443921
    at tryCatcher (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:690:18)
    at _drainQueueStep (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:138:12)
    at _drainQueue (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:131:9)
    at Async._drainQueues (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:147:5)
    at Immediate._onImmediate (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:17:14)
    at process.processImmediate (node:internal/timers:466:21) {
  isCypressErr: true,
  type: 'CLOUD_CANNOT_PROCEED_IN_SERIAL',
  details: undefined,
  messageMarkdown: 'We encountered an unexpected error communicating with our servers.\n' +
    '\n' +
    '`DecryptionError: General JWE must be an object`',
  originalError: undefined,
  stackWithoutMessage: '    at S (<embedded>:4629:437270)\n' +
    '    at <embedded>:4629:443921\n' +
    '    at tryCatcher (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/util.js:16:23)\n' +
    '    at Promise._settlePromiseFromHandler (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:512:31)\n' +
    '    at Promise._settlePromise (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:569:18)\n' +
    '    at Promise._settlePromise0 (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:614:10)\n' +
    '    at Promise._settlePromises (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:690:18)\n' +
    '    at _drainQueueStep (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:138:12)\n' +
    '    at _drainQueue (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:131:9)\n' +
    '    at Async._drainQueues (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:147:5)\n' +
    '    at Immediate._onImmediate (/home/webrunner/.cache/Cypress/12.15.0/Cypress/resources/app/node_modules/bluebird/js/release/async.js:17:14)\n' +
    '    at process.processImmediate (node:internal/timers:466:21)',
  isFatalApiErr: true
} +1m
We encountered an unexpected error communicating with our servers.
DecryptionError: General JWE must be an object
  cypress:server:cypress calling exit 1 +0ms
  cypress:server:cypress about to exit with code 1 +0ms
  cypress:server:browsers browsers.kill called with no active instance +0ms
  cypress:proxy:http:util:prerequests metrics: { browserPreRequestsReceived: 0, proxyRequestsReceived: 0, immediatelyMatchedRequests: 0, unmatchedRequests: 0, unmatchedPreRequests: 0 } +0ms
2023-07-03T19:23:58.541Z cypress:cli child event fired { event: 'exit', code: 1, signal: null }
2023-07-03T19:23:58.541Z cypress:cli Stopping Xvfb
2023-07-03T19:23:58.542Z cypress:cli child event fired { event: 'close', code: 1, signal: null }

I can provide the full logs, however that would need to be sent via DM.

rryter avatar Jul 04 '23 06:07 rryter

Before you spend too much time on this, I have just come accross this: https://docs.cypress.io/faq/questions/cloud-faq#Im-working-with-a-restrictive-VPN-Which-subdomains-do-I-have-to-allow-on-my-VPN-for-Cypress-Cloud-to-work-properly

I have requested all these domains to be whitelisted, and I'll report back if the problem still persists after doing that.

rryter avatar Jul 05 '23 14:07 rryter

Thank you for following up - I was about to go down a rabbit hole. I can do so if placing those domains on a whitelist still doesn't fix your issue.

lmiller1990 avatar Jul 07 '23 01:07 lmiller1990

I'm experiencing the same error.

Context

Running our pipeline on an enterprise GitLab on-prem with:

NodeJS Cypress
16.18.1 12.16.0

Short view of the error:

We encountered an unexpected error communicating with our servers.

RequestError: Error: unable to get local issuer certificate

We will retry 3 more times in 30 seconds...

[ and the retries with the same error ]

Version ugprade

We upgraded from NodeJS 12 and Cypress 9.4.3 to the versions listed above. Everything worked - and still works - with the previous versions.

Based on @mike-plummer's comment above, I configured the cafile and some additional environment variables. Here's the setup:

  • Proxy:
    • npm config set --global proxy <proxy-url>
    • npm config set --global https-proxy <proxy-url>
    • And also via environment variables: HTTP_PROXY, HTTPS_PROXY, NO_PROXY
  • CA:
    • npm config set --global cafile /path/to/cert.pem
    • And also via environment variable export NODE_EXTRA_CA_CERTS=/path/to/cert.pem
  • Certificate validation:
    • Via environment variable: export NODE_TLS_REJECT_UNAUTHORIZED=0
  • Whitelisted Cypress URLs: the ones listed in the documentation

Reproduce it locally

In my local MacBook Pro I don't have any proxy configured, so no need for cafile or any other setting. Locally it works as expected. But as soon as I configure the cafile* in npm I get that error:

$ npm config set --global /path/to/cert.pem
$ npm run test:local-ci
[...]
We encountered an unexpected error communicating with our servers.

RequestError: Error: unable to get local issuer certificate

We will retry 3 more times in 30 seconds...

The test:local-ci NPM script runs a cypress-runner.js file that triggers Cypress from its NodeJS API:

import cypress from 'cypress';

// [...]

cypress.run(runOptions)
  .then(results => { /* ---- */ })
  .catch(err => { /* ---- */ });

/path/to/cert.pem is a valid path to a valid PEM file.

Remove the cafile in the pipeline

I then tried removing the cafile key from the npm config because it's being added in our base Docker image we're running on.

$ npm config delete --global cafile
$ npm run test:ci
[...]
[Successfully connecting to Cypress Cloud and tracking results]

To be honest, I did not spend more time digging why it failed initially or why is it working now. But I hope that helps someone to either fix or identify the issue.

juan-belmonte avatar Jul 12 '23 12:07 juan-belmonte

So it's now working, but you don't know why? Weird.

We upgraded from NodeJS 12 and Cypress 9.4.3 to the versions listed above. Everything worked - and still works - with the previous versions.

If you ever do get to a broken state again, it'd be great if you could verify if it's a specific Cypress version that broke you, or something else.

lmiller1990 avatar Jul 12 '23 23:07 lmiller1990

I don't know why is it working because cacert was needed to access our internal endpoints exposing self-signed certificates, but now Cypress tests succeed when running without cafile configuration.

The fact that Cypress connects to its API endpoint without complaining tells me that our proxy is not adding a new certificate layer between our tests and the Cypress Cloud servers, and hence Cypress connects with no errors.

But considering the initial scenario I described, is Cypress failing because the server certificate it detects does not match the specified CA certificate config — either npm config-related or environment variables — that's found? In other words:

  • The certificate found when connecting to https://api.cypress.io is the real one (issued to cypress.io)
  • The cafile configuration is the one that validates our self-signed certificates
  • They both do not match, and that can cause the error.

In case that's the current behaviour, why would the Cypress binary complain when the real Cypress certificate was found?


On the other hand:

If you ever do get to a broken state again, it'd be great if you could verify if it's a specific Cypress version that broke you, or something else.

You're right: too many moving parts in that upgrade.

juan-belmonte avatar Jul 13 '23 07:07 juan-belmonte

Good discussion here... is there anything we can do on our end to make this experience better? Any actionable work? Or shall we close out this issue as "dunno/cantfix"?

lmiller1990 avatar Jul 14 '23 01:07 lmiller1990

Nothing to do for me, thank you for asking, but I didn't open the issue initially. Still a couple of users in this thread to provide feedback: @Oppon2 and @rryter.

juan-belmonte avatar Jul 14 '23 05:07 juan-belmonte

Was on holidays. On our end the whitelisting of the domains as per https://docs.cypress.io/faq/questions/cloud-faq#Im-working-with-a-restrictive-VPN-Which-subdomains-do-I-have-to-allow-on-my-VPN-for-Cypress-Cloud-to-work-properly did the trick.

rryter avatar Jul 24 '23 12:07 rryter

Great!

I think 3 weeks is enough time. I am going to close this. If there is another issue, let's track in it a new issue. Thanks all!

lmiller1990 avatar Aug 02 '23 22:08 lmiller1990

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information to provide concerning the original issue and we can reopen.

This issue still exists. What is the definition of due to inactivity? There have been multiple issues raised in recent months regarding this.

arafatmusha avatar Aug 30 '23 03:08 arafatmusha

I have not seen other issues, I could be missing them. I think the cause varies from project to project to company to company - might be worth opening a fresh one if you've got a similar problem, so it doesn't get conflated with existing ones.

lmiller1990 avatar Aug 30 '23 23:08 lmiller1990