nx icon indicating copy to clipboard operation
nx copied to clipboard

Postinstall script hangs when installing with pnpm

Open keriati opened this issue 1 year ago • 6 comments
trafficstars

Current Behavior

Postinstall script hangs when installing with pnpm on MacOS.

Expected Behavior

Postinstall script finishes.

GitHub Repo

Every repository using nx, no specific repository is needed.

Steps to Reproduce

  1. run pnpm i with nx in package.json

Nx Report

nx report seems not to work

Failure Logs

❯ NX_VERBOSE_LOGGING=true NX_PERF_LOGGING=true NX_CACHE_PROJECT_GRAPH=false NX_NATIVE_LOGGING=nx node node_modules/nx/bin/post-install.js
TRACE nx::native::workspace::context: workspace_root="/Users/<username>/Projects/<project>"
TRACE nx::native::workspace::context: Initially locking files
TRACE nx::native::workspace::context: Gathering files in /Users/<username>/Projects/<project>
TRACE nx::native::glob: converted globs globs=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"] result=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"]
TRACE nx::native::walker: walking directory="/Users/<username>/Projects/<project>"
TRACE nx::native::walker: walked in 1.607167ms
TRACE nx::native::workspace::files_hashing: Found 0 files
TRACE nx::native::workspace::files_hashing: hashing workspace files in parallel
TRACE nx::native::workspace::files_hashing: hashed workspace files in 131.916µs
TRACE nx::native::workspace::context: hashed and sorted files in 1.923416ms
TRACE nx::native::workspace::files_archive: could not write files archive: No such file or directory (os error 2)
TRACE nx::native::workspace::context: files retrieved files_len=0

Package Manager Version

pnpm 9.3.0

Operating System

  • [X] macOS
  • [ ] Linux
  • [ ] Windows
  • [ ] Other (Please specify)

Additional Information

os: MacOS 15.0 and MacOS 14 node: v20.17.0 pnpm 9.3.0 nx: 19.8.0

Did try node 22, few pnpm versions, always the same result.

Exact same issue is described in this closed ticket: https://github.com/nrwl/nx/issues/27056

keriati avatar Sep 26 '24 20:09 keriati

I have the exact same problem on windows. OS: Windows 11 Pro node: 18.18.2 pnpm: 8.10.0 nx: 19.8.0

gitLinda avatar Sep 27 '24 12:09 gitLinda

Based on comments from previous ticket, i can reproduce the error with this command:

NX_VERBOSE_LOGGING=true NX_PERF_LOGGING=true NX_CACHE_PROJECT_GRAPH=false NX_NATIVE_LOGGING=nx node node_modules/nx/bin/post-install.js

I also get this output:

TRACE nx::native::workspace::context: workspace_root="/Users/<user>/Projects/<project>"
TRACE nx::native::workspace::context: Initially locking files
TRACE nx::native::workspace::files_archive: read archive in 411.459µs
TRACE nx::native::workspace::context: Gathering files in /Users/<user>/Projects/<project>
TRACE nx::native::glob: converted globs globs=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"] result=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"]
TRACE nx::native::walker: walking directory="/Users/<user>/Projects/<project>"
TRACE nx::native::walker: walked in 2.835792ms
TRACE nx::native::workspace::files_hashing: filtered archive files in 667ns
TRACE nx::native::workspace::files_hashing: no additional files to hash
TRACE nx::native::workspace::context: hashed and sorted files in 2.993833ms
TRACE nx::native::workspace::files_archive: write archive in 263.834µs
TRACE nx::native::workspace::context: files retrieved files_len=0
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.29895799999999895
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/package-json' 0.3219169999999991
TRACE nx::native::workspace::context: waiting for files to be available
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/js' 9.766417000000004

keriati avatar Sep 27 '24 12:09 keriati

Compared to machines where this works, my instance is getting stuck on this line:

TRACE nx::native::workspace::context: waiting for files to be available

while on other machines this is instantly followed by:

TRACE nx::native::workspace::context: files are available

This above line never shows up for me, just keeps waiting forever...

keriati avatar Sep 27 '24 13:09 keriati

I found a temporary workaround for me:

killall -9 node rm -r node_modules and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

lennybakkalian avatar Oct 03 '24 07:10 lennybakkalian

I found a temporary workaround for me:

killall -9 node rm -r node_modules and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

Thanks, this worked for me too for the installation step!

Also running nx now works as long as the project stays in /tmp folder.

keriati avatar Oct 03 '24 08:10 keriati

@Cammisuli If you need help to check and try to reproduce this issue, just contact me. (CET)

keriati avatar Oct 18 '24 07:10 keriati

I just hit the same issue, but using yarn. Stuck at waiting for files to be available and but it works if I move my project to /tmp

  • OS: macOS 14.5
  • node 20.18
  • yarn 1.22.22
  • nx 20.0.10

alex-ruehe avatar Nov 14 '24 14:11 alex-ruehe

@keriati any chance you have a global .gitignore? I just did hours of debugging and noticed some debug logs that mentioned my global .gitignore file. After temporarily removing it, everything worked fine.

Not sure what the reason is, since I have zero experience with node and was just trying to get an internal application up and running when I hit the issue.

@Cammisuli maybe this information is helpful?

alex-ruehe avatar Nov 16 '24 19:11 alex-ruehe

@alex-ruehe it seems when I remove my .gitignore file from my home directory (that I use for versioning my dot files), it starts to work normally again.

Maybe this is also somehow connected to the fact that moving the project outside of the home directory makes it also work normally.

I would actually say that the .gitignore in the home directory seems to cause this problem.

keriati avatar Nov 16 '24 19:11 keriati

We tested it also on another machine, and it really seems that the .gitignore file in the user's home directory is causing this issue. We managed to reproduce this issue by adding a .gitignore file in the home directory.

Steps to reproduce:

  • Add .gitignore file in home (~) with the content *. echo "*" > ~/.gitignore
  • Run your install script pnpm i
  • Observe that the process is stuck

Workaround:

  • Remove .gitignore from home directory rm ~/.gitignore
  • Run sudo killall -9 node to stop the stuck processes (or reboot)
  • Run your command again (pnpm i)

keriati avatar Nov 18 '24 07:11 keriati

Thank you for the information. I also have this problem and works fine when I remove .gitignore in home directory

Other solution was

   "pnpm": {
     "overrides": {
       "nx": "19.4.4",

so, I think the cause is introduced right after 19.4.4. Please fix if possible.

awakia avatar Nov 23 '24 05:11 awakia

This happens also, when installing in a Docker environment, where there is no .gitignore present. It's hanging for a couple of minutes until the error below is then displayed, then it continues.

OS: macOS 15.1.1 Node.js: 22.11.0 pnpm: 9.14.2 nx: 20.1.3

These are the postinstall logs from pnpm when executing a docker build The error is Error: Attempted to open socket that exceeds the maximum socket length.

.../node_modules/nx postinstall$ node ./bin/post-install
.../node_modules/nx postinstall:  NX   Failed to start or connect to the Nx Daemon process.
.../node_modules/nx postinstall: Messages from the log:
.../node_modules/nx postinstall: /app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:38
.../node_modules/nx postinstall:         throw new Error([
.../node_modules/nx postinstall:         ^
.../node_modules/nx postinstall: Error: Attempted to open socket that exceeds the maximum socket length.
.../node_modules/nx postinstall: Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue.
.../node_modules/nx postinstall:     at assertValidSocketPath (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:38:15)
.../node_modules/nx postinstall:     at getFullOsSocketPath (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:20:5)
.../node_modules/nx postinstall:     at Object.<anonymous> (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/server/watcher.js:16:44)
.../node_modules/nx postinstall:     at Module._compile (node:internal/modules/cjs/loader:1546:14)
.../node_modules/nx postinstall:     at Object..js (node:internal/modules/cjs/loader:1689:10)
.../node_modules/nx postinstall:     at Module.load (node:internal/modules/cjs/loader:1318:32)
.../node_modules/nx postinstall:     at Function._load (node:internal/modules/cjs/loader:1128:12)
.../node_modules/nx postinstall:     at TracingChannel.traceSync (node:diagnostics_channel:315:14)
.../node_modules/nx postinstall:     at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
.../node_modules/nx postinstall:     at Module.require (node:internal/modules/cjs/loader:1340:12)
.../node_modules/nx postinstall: Node.js v22.11.0
.../node_modules/nx postinstall: More information: /app/.nx/workspace-data/d/daemon.log
.../node_modules/nx postinstall: Done

This is the Dockerfile im am running

FROM node:22-alpine
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
ENV HUSKY=0
ENV COREPACK_ENABLE_AUTO_PIN=0
RUN corepack enable

WORKDIR /app
COPY . .
RUN pnpm install --filter=@scope/service... --frozen-lockfile
EXPOSE 3000
CMD ["pnpm", "nx", "run", "@scope/service:serve"]

Edit

Since the error message mentions to Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue. I logged first to what value it is set. If I didn't make any mistakes, this value is apparently empty

Step 10/12 : RUN echo $NX_SOCKET_DIR
 ---> Running in d8ef0f4cc359

 ---> Removed intermediate container d8ef0f4cc359

So I added the value that was recommended from the error message

ENV NX_SOCKET_DIR=/tmp/nx-tmp

The hang stopped and the build went smooth. Service ran as expected.

yduman avatar Nov 26 '24 17:11 yduman

I ran into this as well. I had a .gitignore in the directory just above my monorepo. Removing it solved the issue.

Edit: I run fedora 39, npm 10.8.2, node 20.18.0 and nx 20.1.2

pontus-aurora avatar Nov 29 '24 14:11 pontus-aurora

hello, unfortunately I have the same problem, tried anything in the thread, but for some reason my npm/pnpm is stuck at post-install any ideas what can I try?

Thanks a lot!

StanDarius24 avatar Dec 02 '24 17:12 StanDarius24

I found a temporary workaround for me:

killall -9 node

rm -r node_modules

and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

This also worked for me. Having it outside of the /Users/<user> seems to do the trick, regardless of the .gitignore. Still not sure what the original issue is here.

MacOs 15.1.1 NX 19.6.5 node 22.3.0 npm 10.8.0

maxtschi avatar Dec 02 '24 18:12 maxtschi

Another data point: I also had a ~/.gitignore in the directory above the one I'm working in - and everything I did with nx would hang - install, run commands, etc. When I removed this file, everything started working again.

Also note that my ~/.gitignore was just *, meaning to ignore everything.. so I wonder if any gitignore that has a generic wildcard causes all files to be ignored, so waiting for files to exist means none will ever exist?

alecf avatar Dec 02 '24 18:12 alecf

Reproducible with npm as well: https://github.com/mfisher87/sscce-nx-with-bare-repo-dotfiles-management-hangs-indefinitely

mfisher87 avatar Dec 13 '24 04:12 mfisher87

I did my best to debug the post-install script. I was able to narrow the hang in my reproduction example above to this line:

https://github.com/nrwl/nx/blob/0d6667d1566a88ebdbae0b65f73188fe95292cc2/packages/nx/src/utils/workspace-context.ts#L59

I got a bit stuck here. I'm not sure what is being called here; there's some Rust code mixed in the "native" module this is imported from, and that's where I find myself a bit out of my depth.

mfisher87 avatar Dec 13 '24 05:12 mfisher87

An somewhat educated guess is that something in nx is "walking" the directory structure and gathering a list of file. Another thread / process is waiting for that list but it never comes... maybe a race condition or bug in the code walking the directory.

It's a bit weird and even "wrong" for nx to need to look outside the top level directory of the project.

pontus-aurora avatar Dec 13 '24 08:12 pontus-aurora

I have the same issue in Gitlab CI environment, I saw the doc says it will turned off in CI mode, guess here might be a bug since it still execute demon logic. os: centos-linux-x64 node: 20.18 pnpm: 9.14.2 nx: 20.1.3

.../.pnpm/[email protected]/node_modules/nx postinstall:  NX   Failed to start or connect to the Nx Daemon process.
.../.pnpm/[email protected]/node_modules/nx postinstall: Messages from the log:
.../.pnpm/[email protected]/node_modules/nx postinstall: /builds/xxx/72781/xxx/node_modules/.pnpm/[email protected]/node_modules/nx/src/daemon/socket-utils.js:38
.../.pnpm/[email protected]/node_modules/nx postinstall:         throw new Error([
.../.pnpm/[email protected]/node_modules/nx postinstall:         ^
.../.pnpm/[email protected]/node_modules/nx postinstall: Error: Attempted to open socket that exceeds the maximum socket length.
.../.pnpm/[email protected]/node_modules/nx postinstall: Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue.

turnerguo avatar Dec 16 '24 07:12 turnerguo

up

andreidiaconescu avatar Jan 14 '25 17:01 andreidiaconescu

Thank you for the information. I also have this problem and works fine when I remove .gitignore in home directory

Other solution was

   "pnpm": {
     "overrides": {
       "nx": "19.4.4",

so, I think the cause is introduced right after 19.4.4. Please fix if possible.

i tried with nx 19.4.4, i use npm, but issue still reproduces for me, practically i am not able to add nx to my project, which i was trying to start using nx on.

andreidiaconescu avatar Jan 15 '25 09:01 andreidiaconescu

I can confirm another datapoint for the home .gitignore. Mine was generated by virtualenv and only includes a wildcard pattern. Removing this file (I just renamed it) solves the problem.

albert118 avatar Jan 29 '25 14:01 albert118

I work around this by cloning the project outside my home dir, then symlinking to that directory from my ~/Projects dir. This way I keep my dotfiles management strategy the same, and can still navigate to my project with cd ~/Projects/myproject instead of remembering the special location I stuck it at.

mfisher87 avatar Jan 29 '25 16:01 mfisher87

I didn't have that kind of issue with nx 20.6.2. But now with 21.2.1 I have it, too. Although I do not have a .gitignore file in my ~/ folder. I am using the node:22.12.0-alpine docker image to test this.

ghost23 avatar Jul 07 '25 13:07 ghost23

Facing the same issue on github actions.

FatalTouch avatar Jul 26 '25 23:07 FatalTouch

Adios nx, entirely too much time wasted.

dev-stratis avatar Aug 13 '25 13:08 dev-stratis

I am on a Mac now and I tried tmp/ and Users/shared, but my monorepo project still hangs on the nx postinstall.

[email protected] node 22.17.1 npm 10.9.2 mac os 15.6

edit:

Now I tried again with tmp/. The only difference being, the first time i git cloned my project through webstorm. then npm install didn't work. Now I did git clone in a terminal and now it worked.

ghost23 avatar Aug 18 '25 09:08 ghost23

Same here

Nx Version:

  • Local: v21.0.3
  • Global: v21.4.1 Node: 22.19.0 NPM: 10.9.3 Mac OS: 15.5 without root privileges

As soon as I copy the project from my home dir (/Users/xxx/dev) into /tmp npm i runs fine.

Also maybe interesting: I once got npm i working in the home dir by renaming the .gitignore, but after that nx serve (angular project) did hang on building the project graph. In the /tmp dir also nx serve works again...

Also moving the project into /Users/Shares/somehing and ln -s that dir into the original home dir works. ONly the files are not allowed to live in the home dir it seems...

JanST123GS avatar Sep 08 '25 11:09 JanST123GS

same here on GitHub Actions with npm

yurayavorski avatar Sep 16 '25 17:09 yurayavorski