node-postgres icon indicating copy to clipboard operation
node-postgres copied to clipboard

npm 7/8+ will install peerDependencies of pg-native by default, pg-native's libpq dep fails to compile on older OSes

Open TysonAndre opened this issue 3 years ago • 2 comments

In npm 6, used by Node.js 14, peerDependencies weren't installed by default In npm 8, used by Node.js 16, peerDependencies were installed by default again. (starting in npm 7)

https://nodejs.org/en/blog/npm/peer-dependencies/ suggests peerDependencies are mainly for use by plugins to depend on the main module https://docs.npmjs.com/cli/v8/configuring-npm/package-json#optionaldependencies

The step of node-gyp installing the peerDependency pg-native's dependency on libpq fails (in Node.js 16) on Linux with the outdated g++ 4.8.5 with g++: error: unrecognized command line option 'std=gnu++14'

  • Comments such as https://github.com/brianc/node-postgres/issues/1645#issuecomment-388244685 suggest pg-native is meant to be used as an optional dependency

  • This is inconvenient if the script running npm install on an older os can't be easily customized with npm install --legacy-peer-deps.

    https://stackoverflow.com/questions/66239691/what-does-npm-install-legacy-peer-deps-do-exactly-when-is-it-recommended-wh

    (In my case, that's probably possible to add to work around this, though the script calling npm install is used in many different applications.) EDIT: npm install --legacy-peer-deps solves my issue, it also removes the dependency from package-lock.json even if it was already installed

~~Note that changing this to an optionalDependency may affect existing applications if they didn't explicitly require pg-native but those applications were previously calling require('pg-native'), so if this does end up getting changed, it may be better for a major release.~~ (never mind, I misunderstood what optionalDependencies were. Those are installed by default.)

Mentioning this because more developers would have started using the new npm 7/8 versions and I didn't see similar questions in the issue tracker


EDIT: Having an OS this outdated is probably an obscure edge case, and this problem may apply to using node.js 16 with any dependency that ran node-gyp rebuild in the post-install script. It may make more sense to close this since there isn't such a thing as a recommended peerDependency, and pg would need to be able to load pg-native and avoid the package manager moving pg up a directory level when deduplicating

TysonAndre avatar Sep 09 '22 20:09 TysonAndre

I already had libpg installed locally the first time I tried to install it (a newer OS than the one where I found this issue), so I missed some issues other than the outdated g++ that could affect users of this library if npm install --legacy-peer-deps isn't used.

Even after using a newer OS and g++ version to fix this and sudo yum install -y python3, I would get a failure at the step which called which pg_config || find /usr/bin /usr/local/bin /usr/pg* /opt -executable -name pg_config -print -quit (returned exit status 1 while in binding.gyp)

EDIT: If I use npm install --legacy-peer-deps in Node.js 16 with npm 8, then subsequent npm ci calls would fail with "npm ci can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync" due to package-lock.json deliberately being created without those peer dependencies. Changing that to npm ci --legacy-peer-deps or adding legacy-peer-deps=true to the local project .npmrc in the application using this (and committing it to version control) successfully works around that

      NOTE: If you create your package-lock.json file by running npm install with flags that can affect the shape of your dependency tree, such as --legacy-peer-deps or --install-links, you must provide the same flags to npm  ci  or      
       you are likely to encounter errors. An easy way to do this is to run, for example, npm config set legacy-peer-deps=true --location=project and commit the .npmrc file to your repo.

TysonAndre avatar Sep 09 '22 21:09 TysonAndre

https://docs.npmjs.com/cli/v8/configuring-npm/package-json#peerdependenciesmeta looks like it can be used to mark this dependency as being optional. npm would attempt to install optional dependencies by default, but allow other dependencies to be installed if that failed to be installed (https://docs.npmjs.com/cli/v8/configuring-npm/package-json#optionaldependencies).

When a user installs your package, npm will emit warnings if packages specified in peerDependencies are not already installed. The peerDependenciesMeta field serves to provide npm more information on how your peer dependencies are to be used. Specifically, it allows peer dependencies to be marked as optional.

For example:

{
  "name": "tea-latte",
  "version": "1.3.5",
  "peerDependencies": {
    "tea": "2.x",
    "soy-milk": "1.2"
  },
  "peerDependenciesMeta": {
    "soy-milk": {
      "optional": true
    }
  }
}

Marking a peer dependency as optional ensures npm will not emit a warning if the soy-milk package is not installed on the host. This allows you to integrate and interact with a variety of host packages without requiring all of them to be installed.

TysonAndre avatar Sep 20 '22 13:09 TysonAndre