cypress icon indicating copy to clipboard operation
cypress copied to clipboard

Published 20 hours ago • verified publisher icon cypress-io

Failure to use NO_PROXY appropriately

Open bdellegrazie opened this issue 3 years ago • 7 comments

Current behavior

In our setup, both environment variables: https_proxy, NO_PROXY and the .npmrc configuration of https_proxy / noproxy are configured.

The particular Cypress download URL is being redirected Nexus via CYPRESS_DOWNLOAD_MIRROR and should not go via the proxy. All of this would work perfectly except: During postinstall the cypress:lib/tasks/download.js task has the following function:

const getProxyForUrlWithNpmConfig = url => {
  return getProxyForUrl(url) || process.env.npm_config_https_proxy || process.env.npm_config_proxy || null;
};

Now getProxyForUrl(url) returns an empty string if it matches NO_PROXY or if there are no variables defined - these are not the same states and should be treated separately. The || treats empty string as falsey and therefore picks up the process.env.npm_config_https_proxy but doesn't handle the corresponding process.env.npm_config_noproxy setting so it incorrectly selects the proxy.

Could you please adjust the function getProxyForUrlWithNpmConfig to correctly handle process.env.npm_config_noproxy and/or treat "nothing defined" differently from proxy explicitly disabled for this domain?

Debug logs

URLs changed!

Environment variables:

    http_proxy = 'http://outbound-proxy.example:3128/'
    https_proxy = 'http://outbound-proxy.example:3128/'
    no_proxy = 'nexus.example,*.example'
    NO_PROXY = 'nexus.example,.example,*.example'
    CYPRESS_DOWNLOAD_MIRROR = 'https://nexus.example/repository/cypress-dist/'

npmrc settings:

proxy=http://outbound-proxy.example:3128/
https_proxy=http://outbound-proxy.example:3128/
noproxy=*.example

Debug logs:

.../[email protected]/node_modules/cypress postinstall: [STARTED] Task without title.
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.892Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.892Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli needed Cypress version: 8.7.0
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli source url https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli downloading cypress.zip to "/tmp/cypress-18030.zip"
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.901Z cypress:cli Downloading package {
.../[email protected]/node_modules/cypress postinstall:   url: 'https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64',
.../[email protected]/node_modules/cypress postinstall:   proxy: 'http://outbound-proxy.example:3128/',
.../[email protected]/node_modules/cypress postinstall:   downloadDestination: '/tmp/cypress-18030.zip'
.../[email protected]/node_modules/cypress postinstall: }
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.945Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.945Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../[email protected]/node_modules/cypress postinstall: 2022-01-06T09:52:36.946Z cypress:cli URL: https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64
.../[email protected]/node_modules/cypress postinstall: Error: tunneling socket could not be established, statusCode=403

Cypress Version

8.7.0

Package Manager

pnpm

Operating system

Linux

Other

No response

bdellegrazie avatar Jan 06 '22 08:01 bdellegrazie

I am experiencing the same problem, though I think it's manifested differently... more like: #4303.. If this doesn't belong here, I'll be happy to create a new issue.

Cypress Version 12.5.1

Package Manager npm / yarn

Operating system MacOs - Monterey

I also think the way the no_proxy is processed (specifically checking for "<-loopback>) makes no sense. It's specifically checking for that loopback string and when it doesn't find it adds "localhost", "127.0.0.1", and "::1" to the no_proxy variable. You'll see in my logs that I already have localhost and 127.0.01 in there. This doesn't really have any affect... just odd behavior.

cypress:server:cypress starting cypress with argv [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args argv array: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args parsed argv options { options: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, 'run-project': 'apps/pim-app-e2e', runProject: 'apps/pim-app-e2e', config: '{"baseUrl":"http://localhost:5000"}', 'config-file': 'cypress.config.ts', configFile: 'cypress.config.ts', env: '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', 'output-path': '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: 'false', 'testing-type': 'e2e', testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0' } } +2ms
  cypress:server:args argv parsed: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { baseUrl: 'http://localhost:5000' }, configFile: 'cypress.config.ts', env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' }, outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true } +5ms
  cypress:server:util:proxy found proxy environment variables { NO_PROXY: 'localhost,127.0.0.1', HTTP_PROXY: 'http://somerandomproxy.com:8000', HTTPS_PROXY: 'http://somerandomproxy.com:8000', no_proxy: 'localhost,127.0.0.1', http_proxy: 'http://somerandomproxy.com:8000', https_proxy: 'http://somerandomproxy.com:8000' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'NO_PROXY' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'HTTP_PROXY' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'HTTPS_PROXY' } +0ms
  cypress:server:util:proxy <-loopback> not found, adding localhost to NO_PROXY +1ms
  cypress:server:util:proxy normalized proxy environment variables { NO_PROXY: 'localhost,127.0.0.1,127.0.0.1,::1,localhost', HTTP_PROXY: 'http://somerandomproxy.com:8000', HTTPS_PROXY: 'http://somerandomproxy.com:8000' } +0ms
  cypress:server:cypress starting in mode run with options { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { e2e: { baseUrl: 'http://localhost:5000' }, env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' } }, configFile: 'cypress.config.ts', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true, proxyServer: 'http://somerandomproxy.com:8000', proxyBypassList: 'localhost,127.0.0.1,127.0.0.1,::1,localhost', projectRoot: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e' } +146ms
    cypress:server:server-base server open +0ms
  cypress:server:server-e2e createServer connecting to server +0ms
  cypress:server:server-base Server listening on  { address: '127.0.0.1', family: 'IPv4', port: 58985 } +25ms
  cypress:server:appdata path: /Users/itsme/Library/Application Support/Cypress/cy/production/proxy +218ms
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +0ms
Cypress could not verify that this server is running:

  > http://localhost:5000

We are verifying this server because it has been configured as your baseUrl.

Cypress automatically waits until your server is accessible before running tests.

We will try connecting to it 3 more times...
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +1m
We will try connecting to it 2 more times...
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +8s
We will try connecting to it 1 more time...

  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [],
  retryIntervals: [ 3000, 3000, 4000 ]
} +10s

As you can see, no_proxy is set correctly, but cypress can't access the server. Below is output from when I unset the proxy information and re-run it, which then successfully connects to the app.

cypress:server:cypress starting cypress with argv [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args argv array: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args parsed argv options { options: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, 'run-project': 'apps/pim-app-e2e', runProject: 'apps/pim-app-e2e', config: '{"baseUrl":"http://localhost:5000"}', 'config-file': 'cypress.config.ts', configFile: 'cypress.config.ts', env: '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', 'output-path': '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: 'false', 'testing-type': 'e2e', testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0' } } +2ms
  cypress:server:args argv parsed: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { baseUrl: 'http://localhost:5000' }, configFile: 'cypress.config.ts', env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' }, outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true } +6ms
  cypress:server:util:proxy found proxy environment variables { NO_PROXY: 'localhost,127.0.0.1', no_proxy: 'localhost,127.0.0.1' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'NO_PROXY' } +0ms
  cypress:server:cypress starting in mode run with options { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { e2e: { baseUrl: 'http://localhost:5000' }, env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' } }, configFile: 'cypress.config.ts', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true, projectRoot: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e' } +177ms
  cypress:server:server-base server open +0ms
  cypress:server:server-e2e createServer connecting to server +0ms
  cypress:server:server-base Server listening on  { address: '127.0.0.1', family: 'IPv4', port: 59724 } +37ms
  cypress:server:appdata path: /Users/itsme/Library/Application Support/Cypress/cy/production/proxy +371ms
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +0ms
cypress:server:open_project open project url http://localhost:5000/__/#/specs/runner?file=src/integration/variables.e2e.ts +2ms

Note: I've only added what I thought to be the relevant logs for both the failing and succeeding case.

mscharp avatar Feb 14 '23 20:02 mscharp

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

cypress-app-bot avatar Aug 14 '23 01:08 cypress-app-bot

The code referenced in the bug description still exists as previously described and the problem still exists. This is not a stale issue.

bdellegrazie avatar Aug 14 '23 07:08 bdellegrazie

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

cypress-app-bot avatar Feb 11 '24 01:02 cypress-app-bot

Still an issue that needs to be addressed

agreedSkiing avatar Feb 12 '24 10:02 agreedSkiing

From what I can see this PR needs to be merged first: https://github.com/Rob--W/proxy-from-env/pull/19 Then, Cypress needs to pick up the new version of that dep and it will hopefully automagically work.

I question whether this codebase should be in the "proxy" business at all. It may find a more appropriate home in the dependency @cypress/request.

WORKAROUND

For anybody else with this problem, our solution was to update our build scripts to prime the cache dir (download and unzip) BEFORE we run Cypress. When Cypress runs it finds everything it needs in the cache and doesn't attempt the download.

ricksbrown avatar Feb 22 '24 01:02 ricksbrown