node-gyp icon indicating copy to clipboard operation
node-gyp copied to clipboard
verified publisher icon nodejs

node-gyp keeps failing unreliably on github windows ci.

Open nodech opened this issue 3 years ago • 5 comments

  • Node Version: 12.x - 18.x

  • Platform: Windows (System Windows_NT 10.0.20348) Github Actions

  • Compiler: VS2022 (17.2.32602.215)

  • Module: https://github.com/nodech/hsd/actions/runs/2515981257

    This issue happens in the Github CI with windows-latest os, everything else is fine. Setup is the matrix of windows with multiple nodejs version from v12 to v18. Because different version of npm are shipped with different versions of node, first step CI does is updates npm so project has latest node-gyp that's shipped with npm. Unfortunately, all of them fail, BUT even worse is they fail unreliably.. Sometimes test will pass, but most of the time it fails. So I am not sure if it's issue with node-gyp or not. But all of the failures happen when looking for common.gypi file. Looking at the code, it seems common.gypi file is not being found at C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\include\\node\\common.gypi and then it fallbacks to what is shown in the output C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi. Here are the example of it failing multiple times:

    • 2 succeed, but others fail: https://github.com/nodech/hsd/actions/runs/2515981257/attempts/2
    • only 1 succeeds: https://github.com/nodech/hsd/actions/runs/2515981257
    • 4 are successfull: https://github.com/nodech/hsd/actions/runs/2515981257/attempts/1

    This happens inconsistently and same test will sometimes pass, mostly fail. Interesting thing to note is that, the dependency itself is building fine: https://github.com/nodech/bcrypto/runs/6939480240, it only fails with hsd which has multiple native dependencies other than bcrypto. I tried to remove all other deps and only leave bcrypto. Switch from matrix to separate tests and etc. Always end up in the similar situation.

Example output for the failure from the Github CI: 
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
npm ERR! code 1
npm ERR! path D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info find Python using Python version 3.9.13 found at "C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http 404 https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp info find VS using VS2022 (17.2.32602.215) found at:
npm ERR! gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'msvs',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build\\config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\16.15.1\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=D:\\a\\hsd\\hsd\\node_modules\\bcrypto',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: C:\Users\runneradmin\AppData\Local\node-gyp\Cache\16.15.1\common.gypi not found (cwd: D:\a\hsd\hsd\node_modules\bcrypto) while reading includes of binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (C:\npm\prefix\node_modules\npm\node_modules\node-gyp\lib\configure.js:261:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Windows_NT 10.0.20348
npm ERR! gyp ERR! command "C:\\hostedtoolcache\\windows\\node\\16.15.1\\x64\\node.exe" "C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! gyp ERR! node -v v16.15.1
npm ERR! gyp ERR! node-gyp -v v9.0.0
npm ERR! gyp ERR! not ok
(...)

nodech avatar Jun 18 '22 13:06 nodech

actions/setup-node@v1 != v3.3.0

actions/checkout should also be upgraded.

cclauss avatar Jun 18 '22 16:06 cclauss

That is main branch which does not have windows tests, that's what I am trying to upgrade and add windows to the mix (already have macos). On the runners you can see branch is update-ci-2: https://github.com/nodech/hsd/blob/update-ci-2/.github/workflows/node.js.yml. Which uses setup-node@v3.

nodech avatar Jun 19 '22 15:06 nodech

So I tried isolating dependencies to see how the failures happen in https://github.com/nodech/ci-test.

Dependencies used for testing (all three are using node-gyp):

Used workflow: https://github.com/nodech/ci-test/blob/main/.github/workflows/nodejs.yml

  • bcrypto alone as dependency - PASS (every build) - https://github.com/nodech/ci-test/actions/runs/2527541403
  • bdb alone as a dependency - PASS (every build) - https://github.com/nodech/ci-test/actions/runs/2527592079
  • bfilter alone as a dependency - PASS (every build) - https://github.com/nodech/ci-test/actions/runs/2527718436
  • bcrypto + bdb as dependencies - INCONSISTENT FAILURES - https://github.com/nodech/ci-test/actions/runs/2527563069
  • bdb + bfilter as dependencies - INCONSISTENT FAILURES - https://github.com/nodech/ci-test/actions/runs/2527621450
  • bcrypto + bfilter as dependencies - INCONSISTENT FAILURES - https://github.com/nodech/ci-test/actions/runs/2527670632

nodech avatar Jun 20 '22 09:06 nodech

I have the same issue since longer times. In my case it is always "noble" which is the reason for the issues... as soon as I use windows-latest runner on GithUb actiopns it fails mainly with Node.js 16!

See https://github.com/Apollon77/hap-controller-node/runs/7120903854?check_suite_focus=true#step:5:53

I use node setup action 3.3.0 and also upgrade node-gyp manually globally.

Apollon77 avatar Jun 29 '22 21:06 Apollon77

Please create a pull request that fails to https://github.com/nodejs/node-gyp/tree/main/.github/workflows so that everyone can see these failures.

cclauss avatar Jul 14 '22 11:07 cclauss

Looks like this is still an issue. I tried using v9.4.0 and I'm still getting this common.gypi not found error. E.g. https://github.com/vercel/hyper/actions/runs/5271238701/jobs/9531822058

LabhanshAgrawal avatar Jun 14 '23 19:06 LabhanshAgrawal

@LabhanshAgrawal looks like it didn't use 9.4.0. From the logs:

gyp ERR! node -v v18.16.0
gyp ERR! node-gyp -v v9.3.1

rithvikvibhu avatar Jun 15 '23 04:06 rithvikvibhu

@LabhanshAgrawal looks like it didn't use 9.4.0. From the logs:

gyp ERR! node -v v18.16.0
gyp ERR! node-gyp -v v9.3.1

Weird, thanks for pointing out, yarn.lock only has the 9.4.0 version. I will check how 9.3.1 is getting used.

LabhanshAgrawal avatar Jun 15 '23 04:06 LabhanshAgrawal

It was picking up v9.3.1 from global packages, setting it up to use the local one solved this. Anyone else facing similar issue on github actions, feel free to check https://github.com/vercel/hyper/pull/7270

LabhanshAgrawal avatar Jun 15 '23 16:06 LabhanshAgrawal

@LabhanshAgrawal++ Thank you for posting this.

rathboma avatar Apr 13 '24 03:04 rathboma