[BUG] npm does not play nicely with pnpm/yarn@2 populated package trees (link targets appear to not resolve correctly)

[GitPopcorn]

In some specific enviroment the code [D:\Program Files\nodejs\node_modules\npm\node_modules@npmcli\arborist\lib\node.js:718:35] will throw a null error when we call npm ls -g or npm i -g because some old dependency links in the node_modules can not be parsed and lost target they linked to.

In my case it's C:\Users\QQ370\AppData\Roaming\npm\node_modules\babel-cli\node_modules\[email protected]@babel-messages, I update the babel-cli and problem solved.

In fact, it is not a bug caused by arborist, but I suggest to add the this.realpath of node to the error logs so that developers could resolve it quickly. Now we got only a single-line message said npm:ERR Cannot read properties of null (reading 'package') and had no clue for what happend. I finally figure it out by debug npm in VSCode, it cost much time for me (I spent lot of time to search online, clear npm caches, change npm root, reinstall the npm/nodejs. downgrade it from v8.5.0 to v7.24.2).

And after I resolved it, I can not recover the enviroment and I have not record the specific version of babel-cli. I'm sorry for that.

There do be someone got the same issue, such as the replier of ISSUE 3194.

Here is the key code:

        if (this.isLink) {
          const target = node.target
          this[_target] = target
          // Errors occured here cuz the target of sub node (node.target) is null
          // But it's obviously that *this* node is a file-link, and the sub node of it should own a target
          this[_package] = target.package
          target.linksIn.add(this)
          // reload edges here, because now we have a target
          if (this.parent) {
            this.parent[_reloadNamedEdges](this.name)
          }
          break
        } else {

Here is the log:

0 verbose cli [
0 verbose cli   'D:\\Program Files\\nodejs\\node.exe',
0 verbose cli   'D:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
0 verbose cli   'ls',
0 verbose cli   '-g'
0 verbose cli ]
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:D:\Program Files\nodejs\node_modules\npm\npmrc Completed in 3ms
6 timing config:load:builtin Completed in 3ms
7 timing config:load:cli Completed in 2ms
8 timing config:load:env Completed in 0ms
9 timing config:load:project Completed in 2ms
10 timing config:load:file:C:\Users\QQ370\.npmrc Completed in 1ms
11 timing config:load:user Completed in 1ms
12 timing config:load:file:C:\Users\QQ370\AppData\Roaming\npm\etc\npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:validate Completed in 1ms
15 timing config:load:credentials Completed in 1ms
16 timing config:load:setEnvs Completed in 0ms
17 timing config:load Completed in 13ms
18 timing npm:load:configload Completed in 14ms
19 timing npm:load:setTitle Completed in 0ms
20 timing config:load:flatten Completed in 4ms
21 timing npm:load:display Completed in 6ms
22 verbose logfile C:\Users\QQ370\AppData\Local\npm-cache\_logs\2022-02-15T05_08_47_962Z-debug-0.log
23 timing npm:load:logFile Completed in 7ms
24 timing npm:load:timers Completed in 0ms
25 timing npm:load:configScope Completed in 0ms
26 timing npm:load Completed in 28ms
27 timing arborist:ctor Completed in 1ms
28 silly logfile start cleaning logs, removing 2 files
29 timing command:ls Completed in 7420ms
30 verbose stack TypeError: Cannot read properties of null (reading 'package')
30 verbose stack     at Link.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:718:35)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:784:20)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:784:20)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:592:22)
30 verbose stack     at Node.set parent [as parent] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:1235:15)
30 verbose stack     at new Node (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:226:17)
30 verbose stack     at Arborist.[newNode] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:322:9)
30 verbose stack     at D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:283:68
30 verbose stack     at async Promise.all (index 0)
30 verbose stack     at async Arborist.[findMissingEdges] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:446:7)
31 verbose cwd C:\WINDOWS\system32
32 verbose Windows_NT 10.0.19042
33 verbose argv "D:\\Program Files\\nodejs\\node.exe" "D:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "ls" "-g"
34 verbose node v16.14.0
35 verbose npm  v8.3.1
36 error Cannot read properties of null (reading 'package')
37 verbose exit 1
38 timing npm Completed in 8073ms
39 verbose code 1
40 error A complete log of this run can be found in:
40 error     C:\Users\QQ370\AppData\Local\npm-cache\_logs\2022-02-15T05_08_47_962Z-debug-0.log

[nlf]

C:\Users\QQ370\AppData\Roaming\npm\node_modules\babel-cli\[email protected]@babel-messages

this path makes it appear as though you globally installed this package with either pnpm or yarn@2 which both use a local store and symlinks to manage their dependencies. npm will not create a directory path like [email protected]@babel-messages in your node_modules as this is not a valid package identifier.

the root issue here is that npm doesn't play nicely with package trees populated by things that are not npm. i'm going to keep this issue open for now, but i'm going to edit the title to show that this is what's going on.

[GitPopcorn]

C:\Users\QQ370\AppData\Roaming\npm\node_modules\babel-cli\[email protected]@babel-messages

this path makes it appear as though you globally installed this package with either pnpm or yarn@2 which both use a local store and symlinks to manage their dependencies. npm will not create a directory path like [email protected]@babel-messages in your node_modules as this is not a valid package identifier.

the root issue here is that npm doesn't play nicely with package trees populated by things that are not npm. i'm going to keep this issue open for now, but i'm going to edit the title to show that this is what's going on.

Thank you, I do have a fuzzy memory of using pnpm long time ago... but I turn to using native npm later and forget it. I still don't really thought this issue as a bug now, npm is not really responsible to recognize the file link, just a simple erorr message to show what cause that is fine enough. Overall, pretty appreciate for your explaination, I can hardly find concrete causes by myself.

[irico3]

I encounter the same error when I run the npm query command in pnpm.

❯ npm query  "*"  -ws myworkspace
npm ERR! Cannot read properties of undefined (reading 'target')

Even if npm is not responsible for supporting symlink tracking up to this point, you may encounter the same problem when adopting the isolated mode. https://github.com/npm/rfcs/blob/main/accepted/0042-isolated-mode.md The above is a direct issue for my project, so I am interested in future developments.