nodejs
`npm pack` crash in Node 23
Version
v23.0.0
Platform
Darwin ....local 23.6.0 Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:44 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_X86_64 x86_64
same in CI
Ubuntu
22.04.5
LTS
Subsystem
No response
What steps will reproduce the bug?
nvm install 23
corepack enable
npm pack
How often does it reproduce? Is there a required condition?
it's only happening on Node 23.0.0. No such issues in Node 18, Node 20 and Node 22.
What is the expected behavior? Why is that the expected behavior?
it should pack
What do you see instead?
(node:9110) ExperimentalWarning: Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
npm warn tarball tarball data for file:/.../ (null) seems to be corrupted. Trying again.
npm warn tarball tarball data for file:/.../ (null) seems to be corrupted. Trying again.
npm error code ENOENT
npm error syscall lstat
npm error path /Users/.../ist/index.cjs
npm error errno -2
npm error enoent ENOENT: no such file or directory, lstat '/.../ist/index.cjs'
npm error enoent This is related to npm not being able to find a file.
npm error enoent
npm error A complete log of this run can be found in: /Users/.../.npm/_logs/2024-10-16T18_41_11_606Z-debug-0.log
⚠️ what's interesting is that ist is actually dist — it's loosing the first letter of the directory name
Additional information
from package.json:
"files": [
"dist",
"migration",
"*.md"
],
from .log file:
0 verbose cli /Users/.../.nvm/versions/node/v23.0.0/bin/node /Users/.../.nvm/versions/node/v23.0.0/bin/npm
1 info using [email protected]
2 info using [email protected]
3 silly config load:file:/Users/.../.nvm/versions/node/v23.0.0/lib/node_modules/npm/npmrc
4 silly config load:file:/Users/.../.npmrc
5 silly config load:file:/Users/.../.npmrc
6 silly config load:file:/Users/.../.nvm/versions/node/v23.0.0/etc/npmrc
7 verbose title npm pack
8 verbose argv "pack"
9 verbose logfile logs-max:10 dir:/Users/.../.npm/_logs/2024-10-16T18_41_11_606Z-
10 verbose logfile /Users/.../.npm/_logs/2024-10-16T18_41_11_606Z-debug-0.log
11 silly logfile start cleaning logs, removing 1 files
12 silly logfile done cleaning log files
13 silly packumentCache heap:4345298944 maxSize:1086324736 maxEntrySize:543162368
14 warn tarball tarball data for file:/Users/.../ (null) seems to be corrupted. Trying again.
15 warn tarball tarball data for file:/Users/.../ (null) seems to be corrupted. Trying again.
16 verbose stack Error: ENOENT: no such file or directory, lstat '/Users/.../ist/index.cjs'
17 error code ENOENT
18 error syscall lstat
19 error path /Users/.../ist/index.cjs
20 error errno -2
21 error enoent ENOENT: no such file or directory, lstat '/Users/.../ist/index.cjs'
22 error enoent This is related to npm not being able to find a file.
22 error enoent
23 verbose cwd /Users/...
24 verbose os Darwin 23.6.0
25 verbose node v23.0.0
26 verbose npm v10.9.0
27 verbose exit -2
28 verbose code -2
29 error A complete log of this run can be found in: /Users/.../.npm/_logs/2024-10-16T18_41_11_606Z-debug-0.log
I can't reproduce. Does the file that it's looking for exist?
$ node -v
v23.0.0
$ cat package.json
{
"name": "package",
"version": "0.0.0"
}
$ npm pack
(node:112161) ExperimentalWarning: Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
npm notice
npm notice 📦 [email protected]
npm notice Tarball Contents
npm notice 49B package.json
npm notice Tarball Details
npm notice name: package
npm notice version: 0.0.0
npm notice filename: package-0.0.0.tgz
npm notice package size: 140 B
npm notice unpacked size: 49 B
npm notice shasum: 9b86ea669576bef9600e739c770b2e5635f75482
npm notice integrity: sha512-sKnSxYIx7NX5u[...]PXtisbzPyqMIA==
npm notice total files: 1
npm notice
package-0.0.0.tgz
I can't reproduce. Does the file that it's looking for exist?
It's looking for /ist/index.cjs while it's dist/index.cjs, @RedYetiDev
The first letter is lost for some reason
Why is it looking for that file? Can you provide your package.json file and directory structure?
I've been able to reproduce now:
$ node -v
v23.0.0
$ cat package.json
{
"name": "package",
"version": "0.0.0"
}
$ tree
.
├── dist
│ └── index.cjs
└── package.json
$ npm pack
(node:114615) ExperimentalWarning: Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
npm warn tarball tarball data for file:/path/ (null) seems to be corrupted. Trying again.
npm warn tarball tarball data for file:/path/ (null) seems to be corrupted. Trying again.
npm error code ENOENT
npm error syscall lstat
npm error path /path/ist/index.cjs
npm error errno -2
npm error enoent ENOENT: no such file or directory, lstat '/path/ist/index.cjs'
npm error enoent This is related to npm not being able to find a file.
npm error enoent
npm error A complete log of this run can be found in: /home/user/.npm/_logs/2024-10-16T18_55_56_185Z-debug-0.log
Looks like a Node.js side issue. I downgraded my npm to v10.8.3 (the version w/ Node.js v22.9.0) and the error still occured. I then downgraded my Node.js to v22.9.0, and kept npm at v10.9.0, and the error no longer occured.
CC @nodejs/npm
Why is it looking for that file? Can you provide your package.json file and directory structure?
I don't know why, but it does not in any other Node.js version.
package.json is here: https://github.com/RobinTail/express-zod-api/blob/test-node23/package.json
directory structure, @RedYetiDev
$ tree -d -L 1
.
├── dist
├── example
├── migration
├── node_modules
├── src
├── tests
└── tools
Here's the stacktrace
READ /.../ist/index.cjs
at Object.lstat (node:fs:1549:15)
at Object.lstat (/node/deps/npm/node_modules/graceful-fs/polyfills.js:309:16)
at [stat] (/node/deps/npm/node_modules/tar/lib/pack.js:181:13)
at [processJob] (/node/deps/npm/node_modules/tar/lib/pack.js:278:19)
at [process] (/node/deps/npm/node_modules/tar/lib/pack.js:232:23)
at [addFSEntry] (/node/deps/npm/node_modules/tar/lib/pack.js:174:18)
at Pack.write (/node/deps/npm/node_modules/tar/lib/pack.js:150:23)
at Pack.add (/node/deps/npm/node_modules/tar/lib/pack.js:129:10)
at addFilesAsync (/node/deps/npm/node_modules/tar/lib/create.js:95:9)
at create (/node/deps/npm/node_modules/tar/lib/create.js:109:3)
at module.exports (/node/deps/npm/node_modules/tar/lib/create.js:39:7)
at /node/deps/npm/node_modules/pacote/lib/dir.js:76:26
Intrestingly, a few calls before this is:
READ /.../dist/index.cjs
at Object.lstat (node:fs:1549:15)
at Object.lstat (/node/deps/npm/node_modules/graceful-fs/polyfills.js:309:16)
at PackWalker.stat (/node/deps/npm/node_modules/ignore-walk/lib/index.js:177:8)
at PackWalker.stat (/node/deps/npm/node_modules/npm-packlist/lib/index.js:192:18)
at /node/deps/npm/node_modules/ignore-walk/lib/index.js:153:14
at Array.forEach (<anonymous>)
at PackWalker.filterEntries (/node/deps/npm/node_modules/ignore-walk/lib/index.js:149:16)
at PackWalker.filterEntries (/node/deps/npm/node_modules/npm-packlist/lib/index.js:173:18)
at PackWalker.onReaddir (/node/deps/npm/node_modules/ignore-walk/lib/index.js:78:14)
at /node/deps/npm/node_modules/ignore-walk/lib/index.js:54:42
at /node/deps/npm/node_modules/graceful-fs/graceful-fs.js:228:16
at FSReqCallback.oncomplete (node:fs:188:23)
Update
The failing call is the call to libpack from libnpmpack in
https://github.com/npm/cli/blob/62c71e5128a01283f97bd62da30ddc673bddda0b/lib/commands/pack.js#L49
More specifically, the call to tarball in https://github.com/npm/cli/blob/62c71e5128a01283f97bd62da30ddc673bddda0b/workspaces/libnpmpack/lib/index.js#L31
Same/Similar issue here that just started occuring today when trying to install renovatebot's pre-commit hook (https://github.com/renovatebot/pre-commit-hooks) which has been working for me up until today.
Worth noting, similar to the ist vs dist mentioned earlier in the thread, if you look closely at this error message, it's looking for a path that includes ode_env-default rather than node_env-default
This is a reminder that "me too" comments only add noise. For everyone considering commenting, please only comment if you have something meaningful to add to the conversation.
If you're experiencing this issue, please 👍 the issue or a comment you agree with. There is no need to comment if you are also affected by the this.
Self-note: We should add pack and other common commands to the test suite of https://github.com/nodejs/citgm/tree/main/test/npm (PRs welcome)
I've made a minimal reproduction:
const walk = require('ignore-walk');
const { resolve, join } = require('path')
const packageDir = resolve('../testing') + '/';
walk({
path: join(packageDir, 'dist'),
parent: {
root: packageDir,
result: new Set([]),
},
}, console.log)
This seems to be an issue with some logic in ignore-walk. I'll continue investigating the root cause.
git bisect is pointing to efbba60e5b8aed95b2413ff4169632bf3605c963 / https://github.com/nodejs/node/pull/54224
$ git bisect log
git bisect start
# status: waiting for both good and bad commits
# bad: [e2242b4e256082e01d8df6a4155ddc80e146c123] 2024-10-16, Version 22.10.0 (Current)
git bisect bad e2242b4e256082e01d8df6a4155ddc80e146c123
# status: waiting for good commit(s), bad commit known
# bad: [20d8b85d3493bec944de541a896e0165dd356345] deps: upgrade npm to 10.9.0
git bisect bad 20d8b85d3493bec944de541a896e0165dd356345
# status: waiting for good commit(s), bad commit known
# bad: [cde6dccb656ad8ad134c2e509c584711156a908c] tools: refactor js2c.cc to use c++20
git bisect bad cde6dccb656ad8ad134c2e509c584711156a908c
# status: waiting for good commit(s), bad commit known
# good: [d2479fa020866f038785fb7373baf2621d7cc1c5] vm: return all own names and symbols in property enumerator interceptor
git bisect good d2479fa020866f038785fb7373baf2621d7cc1c5
# good: [7c58645aca2b53d15dbf698efa43cf934a436d91] lib: move `Symbol[Async]Dispose` polyfills to `internal/util`
git bisect good 7c58645aca2b53d15dbf698efa43cf934a436d91
# bad: [e4692eee75af720640beb557d9279adc0f3e648d] benchmark: --no-warnings to avoid DEP/ExpWarn log
git bisect bad e4692eee75af720640beb557d9279adc0f3e648d
# good: [d38dc99a95fd9dfd634bb938a53871719115fb50] src: add Cleanable class to Environment
git bisect good d38dc99a95fd9dfd634bb938a53871719115fb50
# good: [415905712e59c0d8d7270de27a3b789a121968cd] test: improve test-internal-fs-syncwritestream
git bisect good 415905712e59c0d8d7270de27a3b789a121968cd
# bad: [a65105ec284023960e93b3a66f6661ddd2f4121f] test: adjust tls test for OpenSSL32
git bisect bad a65105ec284023960e93b3a66f6661ddd2f4121f
# good: [f468509cf6f6ebc7d753e8f30686fe0c6b609dc5] zlib: add typings for better dx
git bisect good f468509cf6f6ebc7d753e8f30686fe0c6b609dc5
# bad: [efbba60e5b8aed95b2413ff4169632bf3605c963] path: fix bugs and inconsistencies
git bisect bad efbba60e5b8aed95b2413ff4169632bf3605c963
# good: [c3e1c31baa8f4a59ef49ea0d0429ec55efe2ca81] build: upgrade clang-format to v18
git bisect good c3e1c31baa8f4a59ef49ea0d0429ec55efe2ca81
# first bad commit: [efbba60e5b8aed95b2413ff4169632bf3605c963] path: fix bugs and inconsistencies
cc @huseyinacacak-janea
git bisectis pointing to efbba60 / #54224
path.resolve("/path/");
// v23.0.0: "/path/"
// v22.9.0: "/path"
IMHO path separators shouldn't be preserved since that would make the resolution of identical paths (one with / and one without) return different results. See my revert for more info
I just spent 4 hours of my time trying to debug a new CI setup to publish a package because of this as its affecting all of the {current-, 23-,}bookworm-slim docker images. it initially manifested itself in npm publish as looking into the parent directory for the README.md, which of course failed and there was zero indication as to why it was even leaving the CWD (which was correct in the output after it failed!). I then manually added a "readme": "foobar" to the package.json to get it to skip that, and then it failed in the same broken way as above, removing the d from dist/ when going to collect the files in the directory to create the package. so to help anyone else looking for why npm error enoent ENOENT: no such file or directory, open '<parent of cwd path>/README.md' is happening, this is it. moving to using the node:22-bookworm-slim image immediately worked with no changes needed.
