[BUG] failed optional dependency builds should not prevent global install of packages

[rigor789]

Is there an existing issue for this?

• [X] I have searched the existing issues

This issue exists in the latest npm version

• [X] I am using the latest npm

Current Behavior

With the inclusion of https://github.com/npm/cli/pull/7126 in v10.4 global installs of packages that have transient dependencies on packages like fsevents started failing.

Example Logs

npm ERR! code 1
npm ERR! path /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | darwin | arm64
npm ERR! gyp info find Python using Python version 3.9.6 found at "/Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3"
npm ERR! gyp info spawn /Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/Library/Caches/node-gyp/21.7.2/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/Users/rigor789/Library/Caches/node-gyp/21.7.2',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/Users/rigor789/Library/Caches/node-gyp/21.7.2/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.<anonymous> (/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/lib/configure.js:297:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:519:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Darwin 23.4.0
npm ERR! gyp ERR! command "/Users/rigor789/.nvm/versions/node/v21.7.2/bin/node" "/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! gyp ERR! node -v v21.7.2
npm ERR! gyp ERR! node-gyp -v v10.1.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/rigor789/.npm/_logs/2024-04-09T09_05_41_833Z-debug-0.log

Expected Behavior

Global installs should not fail. fsevents is an optional peerDependency, and installing it should not trigger a rebuild, nor fail on unsupported systems (like Linux/Windows).

Steps To Reproduce

1. install npm 10.4 or newer
2. run npm i -g nativescript
3. the install will fail

To confirm the linked change breaks it:

1. open node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js of your global packages
2. edit line 1235 and comment out // fullMetadata: true,
3. re-run: npm i -g nativescript
4. the install works as expected.

Environment

• npm: 10.4.0 (or newer)
• Node.js: v21.7.2
• OS Name: macOS Sonoma 14.4.1
• System Model Name: MacBook Pro
• npm config:

; "user" config from /Users/rigor789/.npmrc

//registry.npmjs.org/:_authToken = (protected)

; node bin location = /Users/rigor789/.nvm/versions/node/v21.7.2/bin/node
; node version = v21.7.2
; npm local prefix = /Users/rigor789/Code/
; npm version = 10.4.0
; cwd = /Users/rigor789/Code/
; HOME = /Users/rigor789
; Run `npm config ls -l` to show all defaults.

[rigor789]

If it's helpful, here are the manifests before/after including fullMetadata: true

Without `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  devDependencies: { 'node-gyp': '^9.4.0' },
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  os: [ 'darwin' ],
  hasInstallScript: true,
  _id: '[email protected]',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

With `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  description: 'Native Access to MacOS FSEvents',
  main: 'fsevents.js',
  types: 'fsevents.d.ts',
  os: [ 'darwin' ],
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  scripts: {
    clean: 'node-gyp clean && rm -f fsevents.node',
    build: 'node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean',
    test: '/bin/bash ./test.sh 2>/dev/null',
    prepublishOnly: 'npm run build',
    install: 'node-gyp rebuild'
  },
  repository: { type: 'git', url: 'git+https://github.com/fsevents/fsevents.git' },
  keywords: [ 'fsevents', 'mac' ],
  contributors: [
    { name: 'Philipp Dunkel', email: '[email protected]' },
    { name: 'Ben Noordhuis', email: '[email protected]' },
    { name: 'Elan Shankar', email: '[email protected]' },
    { name: 'Miroslav Bajtoš', email: '[email protected]' },
    { name: 'Paul Miller', url: 'https://paulmillr.com' }
  ],
  bugs: { url: 'https://github.com/fsevents/fsevents/issues' },
  homepage: 'https://github.com/fsevents/fsevents',
  devDependencies: { 'node-gyp': '^9.4.0' },
  gypfile: true,
  gitHead: '2db891e51aa0f2975c5eaaf6aa30f13d720a830a',
  _id: '[email protected]',
  _nodeVersion: '18.17.1',
  _npmVersion: '9.6.7',
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  _npmUser: { name: 'pipobscure', email: '[email protected]' },
  directories: {},
  maintainers: [
    { name: 'pipobscure', email: '[email protected]' },
    { name: 'paulmillr', email: '[email protected]' }
  ],
  _npmOperationalInternal: {
    host: 's3://npm-registry-packages',
    tmp: 'tmp/fsevents_2.3.3_1692635062642_0.01197687980904627'
  },
  _hasShrinkwrap: false,
  _time: '2023-08-21T16:24:22.854Z',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

[wraithgar]

This appears to have been working by accident before. The minified packument does not have scripts which likely means that any package that required a build step would not work globally if we reverted that line.

This is still a bug, but the bug is that this should not be failing in global mode. Local mode gracefully catches this:

npm info run [email protected] install node_modules/fsevents node-gyp rebuild
npm info run [email protected] install { code: 1, signal: null }
npm verb reify failed optional dependency /Users/wraithgar/Development/npm/scratch/optional/node_modules/fsevents
npm info run [email protected] postinstall node_modules/nativescript node postinstall.js
npm info run [email protected] postinstall { code: 0, signal: null }

[rigor789]

Thanks for confirming, indeed that sounds more reasonable as an explanation - I wouldn't expect optional dependencies to fail a global install.

I do wonder when that install: 'node-gyp rebuild' is being added however, because it's not in the published package.json of fsevents:

"scripts": {
  "clean": "node-gyp clean && rm -f fsevents.node",
  "build": "node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean",
  "test": "/bin/bash ./test.sh 2>/dev/null",
  "prepublishOnly": "npm run build"
},

(from https://www.npmjs.com/package/fsevents?activeTab=code)

[wraithgar]

It's added automatically by npm during publish because it detects no existing install script but it finds a binding.gyp file. You can see an old issue where folks had similar problems here: https://github.com/npm/cli/issues/5234

[koiker]

Got the same issue. The fix was to downgrade npm: npm i -g npm@~10.3