[BUG] devDependencies of project with npm-shrinkwrap.json are getting installed

[RandomByte]

Is there an existing issue for this?

• [X] I have searched the existing issues

This issue exists in the latest npm version

• [X] I am using the latest npm

Current Behavior

When installing a package (npm install <package name>) from the npm registry that contains an npm-shrinkwrap.json, dependencies listed with "dev": true are installed unexpectedly.

Expected Behavior

devDependencies of the published package should not be installed when consuming the package.

Steps To Reproduce

Given: Node.js v16.13.2 and npm v8.3.2

I created an example project to showcase this issue: https://github.com/RandomByte/npm-shrinkwrap-test

1. git clone [email protected]:RandomByte/npm-shrinkwrap-test.git && cd npm-shrinkwrap-test
2. npm install
3. Check for known devDependency "ava" and notice that it has been installed:

   npm ls -a | grep 'ava'
   > ├─┬ [email protected] extraneous
   

4. npm prune --production removes the ava package. But a following npm install will install it again

It seems that after installing @ui5/cli@next in the empty project, its devDependencies where installed as normal dependencies. This is unexpected since a consumer of the package does not need its devDependencies.

In the past (with npm v6) we worked around this issue by executing npm prune --production followed by npm shrinkwrap during publishing of the @ui5/cli package. This basically removed all devDependencies from the npm-shrinkwrap.json (see https://github.com/SAP/ui5-tooling/issues/283#issuecomment-599985109). But this does not seem to work with npm v8 anymore, as the npm-shrinkwrap.json is not being updated after the prune took place.

I think the fundamental issue is present since at least npm v6. Relevant issues are:

• https://github.com/npm/cli/issues/1036
• https://github.com/npm/cli/issues/1878
• https://github.com/npm/cli/issues/1113
  • This issue has actually been fixed: In the project above npm install --production does not install the dependency's devDependencies. However, I would expect this exact behavior from npm install (without the --production flag) since I don't see a point in installing transitive devDependencies

Environment

• npm: v8.3.2
• Node.js: v16.13.2
• OS Name: macOS 12.1 (Monterey)
• System Model Name: MacBook Pro
• npm config:

registry = "https://registry.npmjs.com/" 

[bkendall]

I'm going to +1 this via comment as well for visibility: I just worked around this in part of our build process for firebase-tools; it caused our prebuilt binaries to bloat from 130MB to 800MB+ and had to work pretty hard to find a workaround (thankfully we found one).

[nlf]

confirmed bug, we're ignoring the dev flag in the npm-shrinkwrap.json, installing the dev dependencies and flagging them as extraneous when we write the package-lock.json.

i confirmed the npm-shrinkwrap.json flags the relevant deps as dev.

as noted, npm prune --production removes the extraneous deps from node_modules but not from the package-lock, so a follow up npm install puts them right back.

as a further note, npm install --only=prod does skip reifying the dev deps, as does npm install --omit=dev so we are internally identifying them correctly, just for some reason when we reify a dep with a shrinkwrap we're not applying the default rules for dev dependencies.

[bkendall]

@nlf any chance that this can get prioritized?

Another interesting issue that comes with this is that npm audit (which probably shouldn't be running when a module is installed globally in our case, but that's another bug) checks all the dev dependencies that were installed too, which creates more noise for end users of our shrinkwrap'd CLI.

[haines]

It's also quite frustrating with Dependabot updates, because Dependabot uses npm install --package-lock-only which does not result in the shrinkwrapped devDependencies being installed. This creates churn with Dependabot removing the extraneous package-lock.json entries and then local npm install reintroducing them.

[bkendall]

This is something that we're scratching our head to solve. Installing all our dev dependencies when installing firebase-tools globally takes up a ton of space, and it'd be nice to not do that. We've come across other options like clean-publish but working around this issue in npm seems like not the correct way to do it long term.

It'd be really appreciated if this could be addressed soon! Thanks!

[bkendall]

Another week, another person surprised when dev dependencies are installed and it triggers other audit warnings too. It would be really great to get this fixed.

https://github.com/firebase/firebase-tools/issues/4586

[bkendall]

Once again, another person has escalated an issue to me where the dev dependencies are being installed. Would love to see this addressed soon.

[cupidchan]

Is there any timeline for this issue? It's pretty confusing to see so many irrelevant errors/warnings.

[antonstefer]

Commenting for visibility. Not sure if you're responsible @nlf but this deserves some attention.

[cupidchan]

This issue has been opened for a while now and we still see those irrelevant warnings and errors. Can this be prioritized and fixed in the near future? Thanks!

[bkendall]

I've given up on this ever being fixed and worked around it using clean-publish. I'd like to get back to using npm directly, but cutting the installed size by 75% just cannot be a benefit that I can wait on any longer.

[RandomByte]

Update: This can still be reproduced as described with Node v20.11.1 and npm v10.2.4