std::bad_alloc starting from 20.12.0

[bolt-juri-gavshin]

Version

20.12.0

Platform

x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Build the following Dockerfile

FROM node:20.12.0-bookworm
WORKDIR /app
RUN echo 'foreground-scripts = true' >> .npmrc
RUN npm install [email protected] --force --loglevel=verbose

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

Same as in 20.11.1:

<NPM fetch rows>
#7 3.613 npm info run [email protected] install node_modules/rocksdb node-gyp-build
#7 3.613 
#7 3.613 > [email protected] install
#7 3.613 > node-gyp-build
#7 3.613 
#7 3.711 npm info run [email protected] install { code: 0, signal: null }
#7 3.722 
#7 3.722 added 12 packages in 3s
#7 3.722 
#7 3.722 5 packages are looking for funding
#7 3.722   run `npm fund` for details
#7 3.722 npm notice 
#7 3.723 npm notice New minor version of npm available! 10.2.4 -> 10.5.1
#7 3.723 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.5.1>
#7 3.723 npm notice Run `npm install -g [email protected]` to update!
#7 3.723 npm notice 
#7 3.723 npm verb exit 0
#7 3.723 npm info ok 
#7 DONE 3.9s

What do you see instead?

<NPM fetch rows>
#7 2.704 npm info run [email protected] install node_modules/rocksdb node-gyp-build
#7 2.704 
#7 2.704 > [email protected] install
#7 2.704 > node-gyp-build
#7 2.704 
#7 2.806 terminate called after throwing an instance of 'std::bad_alloc'
#7 2.806   what():  std::bad_alloc
#7 2.806 Aborted
#7 2.806 
#7 2.847 gyp info it worked if it ends with ok
#7 2.848 gyp verb cli [
#7 2.848 gyp verb cli '/usr/local/bin/node',
#7 2.848 gyp verb cli '/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js',
#7 2.848 gyp verb cli 'rebuild'
#7 2.848 gyp verb cli ]
#7 2.848 gyp info using [email protected]
#7 2.848 gyp info using [email protected] | linux | x64
#7 2.853 gyp verb clean removing "build" directory
#7 2.882 gyp verb find Python Python is not set from command line or npm configuration
#7 2.883 gyp verb find Python Python is not set from environment variable PYTHON
#7 2.883 gyp verb find Python checking if "python3" can be used
#7 2.883 gyp verb find Python - executing "python3" to get executable path
#7 2.897 gyp verb find Python - executable path is "/usr/bin/python3"
#7 2.897 gyp verb find Python - executing "/usr/bin/python3" to get version
#7 2.912 gyp verb find Python - version is "3.11.2"
#7 2.915 gyp info find Python using Python version 3.11.2 found at "/usr/bin/python3"
#7 2.915 gyp verb get node dir no --target version specified, falling back to host node version: 20.12.0
#7 3.006 
#7 3.007 gyp verb install input version string "20.12.0"
#7 3.007 gyp verb install installing version: 20.12.0
#7 3.007 gyp verb install --ensure was passed, so won't reinstall if already installed
#7 3.008 gyp verb install version not already installed, continuing with install 20.12.0
#7 3.008 gyp verb ensuring devDir is created /root/.cache/node-gyp/20.12.0
#7 3.008 gyp verb created devDir /root/.cache
#7 3.008 gyp http GET https://nodejs.org/download/release/v20.12.0/node-v20.12.0-headers.tar.gz
#7 3.094 gyp http 200 https://nodejs.org/download/release/v20.12.0/node-v20.12.0-headers.tar.gz
<VERY LONG COMPILATION>

Additional information

Not sure, whether the bug is in NodeJS or rocksdb or node-gyp-build, but this code worked fine in NodeJS 16, 18 and 20.11.

[z-x]

Not sure if this is the place, but I am seeing an error related to node-gyp from 20.12.0 as well. Basically

binding.gyp not found (cwd: /__t/node/21.7.2/x64/lib/node_modules/@platformos/pos-cli/node_modules/fsevents) while trying to load binding.gyp

while installing a package that was working before without any changes. Falling back to Node 20.11 fixes this problem.

I can provide more details if needed, but the package in question is https://github.com/mdyd-dev/pos-cli and switching Node versions is everything one needs to reproduce it. Fails only on Linux (and possibly OSX), I am using mcr.microsoft.com/playwright:v1.42.1-jammy, seems to be working on Windows.

To reproduce on Node 20.12.1:

1. Use the Docker image https://hub.docker.com/_/microsoft-playwright
2. Run npm i -g @platformos/pos-cli

UPDATE: I tried updating all of the dependencies to see if that changes anything but it doesn't.

[z-x]

To update this issue as I did some further investigating. It seems that the issue is not strictly corresponding to Node, but to npm version that Node comes with. I have tested npm i -g @platformos/pos-cli on Node 22.12.2 and [email protected] and above and it does not work. But when I've downgraded npm to 10.3.0 everything works, even on 22.12.2.

@bolt-juri-gavshin - would you be able to confirm?

[bolt-juri-gavshin]

Downgrading to NPM 10.3.0 helps in "my real-world usage scenario", even if NodeJS 20.12.2 is used (on linux/x86_64 and linux/arm64). For some reason, my example Dockerfile still fails, even if NPM is downgraded to 10.3.0.

[z-x]

Also, pnpm does not have any problems here.

[z-x]

Ok, so in my case the problematic package seems to be chokidar. I've replaced it with @parcel/watch and it seems to be working. The downside is that we were using LiveReload as well that directly depends on chokidar and the only working solution I've found was to remove it from the package. Nothing else was working.

[bolt-juri-gavshin]

Update: It looks like NodeJS in 20.14.0 (NPM 10.7.0) the problem is fixed. This Dockerfile works now:

FROM node:20.14.0-bookworm
WORKDIR /app
RUN echo 'foreground-scripts = true' >> .npmrc
RUN npm install [email protected] --force --loglevel=verbose

[z-x]

For me nothing changed in terms of my package, but I have found a fix.

npm i fsevents@latest -f --save-optional

Fixes the problem for some reason, thus I think there is something wrong in newest npm with managing optional dependencies, but I am not skilled enough in Node to investigate further.