pnpm icon indicating copy to clipboard operation
pnpm copied to clipboard

postinstall script not run if project restored

Open MikeMcC399 opened this issue 10 months ago • 23 comments

Verify latest release

  • [x] I verified that the issue exists in the latest pnpm release

pnpm version

10.4.1

Which area(s) of pnpm are affected? (leave empty if unsure)

CLI

Link to the code that reproduces this issue or a replay of the bug

https://github.com/MikeMcC399/cypress-test-tiny/tree/pnpm-10-corepack-install

Reproduction steps

Ubuntu 24.04.2 LTS, Node.js 22.14.0 LTS using [email protected]

corepack enable pnpm
rm -rf ~/.local/share/pnpm # clear store for fresh start

## First installation
git clone --branch pnpm-10-corepack-install https://github.com/MikeMcC399/cypress-test-tiny
cd cypress-test-tiny
pnpm install # install dependencies
pnpm cypress verify # check that all components are installed
pnpm cypress cache clear # clear cache installed by cypress postinstall

## Remove cloned repo
cd ..
rm -rf cypress-test-tiny

## Second restored installation
git clone --branch pnpm-10-corepack-install https://github.com/MikeMcC399/cypress-test-tiny # clone repo a second time
cd cypress-test-tiny
pnpm install # install dependencies
pnpm cypress verify # fails - cypress cache not installed

Describe the Bug

If an npm package with an allowed postinstall script is removed and reinstalled, the postinstall script is not run for the re-installation.

The package is allowed through pnpm.onlyBuiltDependencies configuration record of package.json.

Note: this is not a regression from pnpm 9.x. The same problem already previously existed, except all scripts were allowed anyway.

Expected Behavior

Even if a package is already present in pnpm's store, running pnpm install should always run the postinstall script (if allowed).

Which Node.js version are you using?

v22.14.0 LTS

Which operating systems have you used?

  • [ ] macOS
  • [ ] Windows
  • [x] Linux

If your OS is a Linux based, which one it is? (Include the version if relevant)

Ubuntu 24.04.2 LTS

MikeMcC399 avatar Feb 17 '25 09:02 MikeMcC399

For comparison, Yarn 4 pnp does run the postinstall script if a project is reinstalled.

MikeMcC399 avatar Feb 17 '25 09:02 MikeMcC399

pnpm uses a side-effects cache by default. This means that a package with a postinstall script is built one time on a machine and written to cache. Next time the cached prebuild version is installed.

zkochan avatar Feb 17 '25 12:02 zkochan

What is cypress doing that it needs to run a postinstall that cannot be cached? Does it make changes to files outside the cypress package directory or what?

As far as I remember the husky package did some actions that couldn't be cached. They were creating git hooks in the current repo. They then changed it from a postinstall hook to a requirement to run husky setup from the dependent projects own prepare script: https://github.com/pnpm/pnpm/blob/0332bfcd81ca22076e303d17dbb05bb2b263effb/package.json#L9

zkochan avatar Feb 17 '25 13:02 zkochan

@zkochan

Let me look at the side-effects cache setting to see if it solves the issue.

Cypress uses postinstall to install its Electron app. This is indeed cached, however users have reported issues in the past in GitHub Actions where the pnpm store is cached separately to the Cypress cache and they can get out of sync, especially if there are parallel jobs running.

Yes it does make changes outside the Cypress package directory.

It's described in https://docs.cypress.io/app/references/advanced-installation#Binary-cache and https://docs.cypress.io/app/continuous-integration/overview#Caching

MikeMcC399 avatar Feb 17 '25 13:02 MikeMcC399

@zkochan

Thank you very much for your help!

Adding

side-effects-cache=false

to the project's .npmrc resolves the issue. I will pass this on so that the information can be added to the Cypress documentation on https://docs.cypress.io/app/get-started/install-cypress

MikeMcC399 avatar Feb 17 '25 13:02 MikeMcC399

That's not a good solution. You will disable it for all dependencies.

zkochan avatar Feb 17 '25 15:02 zkochan

@zkochan

That's not a good solution. You will disable it for all dependencies.

Do you have a better solution? In the Cypress projects that I have seen, it is unusual for other dependencies to also have a postinstall script, so I don't think it would be a real disadvantage in practice if it is disabled.

MikeMcC399 avatar Feb 17 '25 16:02 MikeMcC399

I see a couple of possible solutions.

We could check if there are any changes to the files of the package after running postinstall. If no, we probably can't cache it. But this might not be reliable.

Another option would be to allow specific packages to opt-out from side-effects cache. Maybe some field in package.json, like "pnpm.bypassSideEffectsCache=true". Related issue: https://github.com/pnpm/pnpm/issues/5271

zkochan avatar Feb 17 '25 19:02 zkochan

@zkochan

I see a couple of possible solutions.

We could check if there are any changes to the files of the package after running postinstall. If no, we probably can't cache it. But this might not be reliable.

Another option would be to allow specific packages to opt-out from side-effects cache. Maybe some field in package.json, like "pnpm.bypassSideEffectsCache=true". Related issue: #5271

  • Allowing specific packages to opt out would be ideal. Thanks for the link to the related issue #5271! For the moment I think that using side-effects-cache=false is going to be OK in most cases. Thanks for pointing out that it could be negative for other packages though. If we add the option to the Cypress documentation then a corresponding note about the side-effects could be added as well.

MikeMcC399 avatar Feb 17 '25 19:02 MikeMcC399

We had a similar issue with puppeteer which runs a a post install scripts to install a chrome instance to $HOME/.cache/puppeteer. Beside adding

side-effects-cache=false

to .npmrc

We also had to add:

"pnpm": {
        "onlyBuiltDependencies": [
            "puppeteer"
        ]
    }

to package.json.

ablomq avatar Mar 07 '25 08:03 ablomq

We had a similar issue with puppeteer which runs a a post install scripts to install a chrome instance to $HOME/.cache/puppeteer. Beside adding

side-effects-cache=false

to .npmrc

We also had to add:

"pnpm": {
        "onlyBuiltDependencies": [
            "puppeteer"
        ]
    }

to package.json.

I encountered it today as well, but mine was an Electron project.I just needed to add the part you mentioned below. Actually, after running pnpm i, there were warning messages. I had to manually execute pnpm approve-builds, select which packages need to be executed manually, and then generate the configuration items into the package.json file.

"pnpm": {
    "onlyBuiltDependencies": [
      "electron"
    ]
  }

jiawei397 avatar Mar 19 '25 10:03 jiawei397

onlyBuiltDependencies is a slightly different topic If you don't have this defined, then the postinstall script will never run

This topic is only about the postinstall script not running after restoring a project and, as discussed above, this is a caching issue.

MikeMcC399 avatar Mar 19 '25 12:03 MikeMcC399

My pnpm approve-builds command only showed esbuild.. But I want cypress to get installed. For some reason pnpm install is NO longer installing Cypress.

My current workaround is to run:

pnpm exec cypress install

After pnpm install. I have no clue how to fix it. I tried adding cypress to the onlyBuiltDependencies list. Of course that doesn't work.

Cypress documentation is not clear and only mention "allowlisting": https://docs.cypress.io/app/get-started/install-cypress#pnpm-Configuration

[email protected] and above require allowlisting cypress. This enables Cypress to execute its postinstall script so it can install the Cypress binary into the binary cache. Refer to the pnpm configuration file documentation for additional information. [email protected] introduced the CLI add option --allow-build to add the allowed build configuration directly through the command line.

Well I can tell you allow build doesn't help during a simple pnpm install. Help?

melroy89 avatar Apr 09 '25 20:04 melroy89

@melroy89

Which version of pnpm are you using? Are you having this problem locally or in a CI workflow? If CI, which provider is it?

Have you set side-effects-cache=false ? If yes, then try deleting the node_modules directory and re-running pnpm install

If that doesn't work, I suggest opening a new issue to gather more details about your configuration.

MikeMcC399 avatar Apr 10 '25 05:04 MikeMcC399

  • I logged a related regression in https://github.com/pnpm/pnpm/issues/9394

MikeMcC399 avatar Apr 10 '25 17:04 MikeMcC399

@melroy89

Please let us know if your issue is perhaps resolved with the previous information or if you still need assistance.

MikeMcC399 avatar Apr 10 '25 17:04 MikeMcC399

Thanks for replying. I can also still answer first your earlier questions.

Which version of pnpm are you using?

The latest:

pnpm --version            
10.8.0

Are you having this problem locally or in a CI workflow? If CI, which provider is it?

Actually both. I can reproduce the lack of installing Cypress locally on my Linux system as well as in CI/CD. For CI/CD I user a GitLab / GitLab runner with Docker. cypress/browsers:node-22.14.0-chrome-135.0.7049.84-1-ff-137.0.1-edge-135.0.3179.54-1 to be precise.

Locally I can removed my ~/.cache/Cypress folder first, from npm (the old times). And trying to trigger a fresh install and install of Cypress by running pnpm install (yes cypress packages is part of package.json dev dep of course). But like I said, it won't install Crypress (post-install I guess?).

Have you set side-effects-cache=false ? If yes, then try deleting the node_modules directory and re-running pnpm install

I now added side-effects-cache=false to .npmrc.

Ah yes, then I removed the node_modules dir and then re-run it. That indeed seems to work.

It does feel like a workaround. Especially since I now need to create a new .npmrc file, despite using pnpm. lol. It just feels wrong. So writing the side-effects-cache=false to pnpm-workspace.yaml would be the ideal solution? It still sounds like a workaround to be honest.

What is the ideal solution in the future? Maybe an additional install flag or? @zkochan this is really a big issue, I think pnpm need to be fixed here.


EDIT: Even with this side effects cache enabled, I would have expected post-install script to run if Cypress isn't yet installed on the first install. right? Maybe it won't try it again after a second or third pnpm install.

EDIT EDIT: Also setting the following config in pnpm-workspace.yaml is NOT sufficient enough:

onlyBuiltDependencies:
  - cypress

melroy89 avatar Apr 10 '25 17:04 melroy89

@melroy89

You can consider side-effects-cache=false a workaround, however I would expect this to need to stay with us for quite a while.

The ideal situation would be if pnpm defined a key that Cypress could add to its package.json to say it is not compatible with side effects caching, then pnpm could take account of that without any user intervention being necessary.

I think you saw that there is an issue (https://github.com/pnpm/pnpm/issues/9394) with using pnpm-workspace.yaml for the setting. ~~We'll need to wait for feedback on that point.~~ Edit: This issue is now resolved.

In my tests, if Cypress hasn't been previously installed and is not cached then the postinstall script will run (assuming it is listed in onlyBuiltDependencies or similar).

MikeMcC399 avatar Apr 10 '25 18:04 MikeMcC399

@melroy89

Cypress documentation is not clear and only mention "allowlisting"

  • I've submitted a suggestion to improve the Cypress documentation https://github.com/cypress-io/cypress-documentation/issues/6147. I agree that it shouldn't be this difficult to work out what to do!

MikeMcC399 avatar Apr 11 '25 07:04 MikeMcC399

@melroy89

The Cypress documentation https://docs.cypress.io/app/get-started/install-cypress#pnpm-Configuration has been updated to include instructions to disable the pnpm side effects cache.

MikeMcC399 avatar Apr 14 '25 19:04 MikeMcC399

@melroy89

The Cypress documentation https://docs.cypress.io/app/get-started/install-cypress#pnpm-Configuration has been updated to include instructions to disable the pnpm side effects cache.

Thanks for the update. Yeah, I can indeed confirm the workaround works. Lets hope a final fix in pnpm will be released.

melroy89 avatar Apr 15 '25 00:04 melroy89

  • Also related to https://github.com/pnpm/pnpm/issues/5002

MikeMcC399 avatar Apr 15 '25 07:04 MikeMcC399

Thanks for all the information so far, I would like to provide another data point. I am using Prisma which relies on their postinstall script to create client folder (.prisma/client), and this can't be cached by pnpm at the moment (v10.12.1). This has caused some confusions before (issues).

As others already pointed out, I think either allowing the consumers to disable side effects cache by package, or allowing the consumed packages to indicate they are not cachable by pnpm would be helpful. Maybe a combination of both to provide the most flexibility, and start with the consumer side as it is more controllable.

CHC383 avatar Jun 15 '25 00:06 CHC383

  • Closing this issue, as the reason for the problem was already clarified (side effects cache).
  • Moving the discussion about a feature enhancement instead to https://github.com/pnpm/pnpm/issues/9949
  • Note that there is already a more general feature description in https://github.com/pnpm/pnpm/issues/5002

MikeMcC399 avatar Sep 08 '25 07:09 MikeMcC399