[regression] promotion of directory targets has trouble with directories starting in @

[ejgallego]

trafficstars

Dear dune devs,

updating to Dune 3.15, I've found the following problem with this simple rule (that used to work):

(rule
 (targets
   (dir node_modules))
 (deps package.json)
 (mode promote)
 (action
  (bash "npm i")))

it seems in 3.15.3, dune chokes with:

Error: Cannot promote files to "node_modules/@babel/parser".
Reason: opendir(node_modules/@babel/parser): No such file or directory
-> required by _build/default/node_modules

File is correctly present in _build dir.

Any package.json file that depends on a package starting with @ seems to do the trick, for example:

{
  "name": "test",
  "version": "0.17.0",
  "type": "module",
  "description": "NPM test",
  "main": "index.js",
  "dependencies": {
    "@octokit/core": "^4.2.1"
  }
}

Bisect points to b1c339b868e00fd5a1a3e72db90e57b4f3285208 , I've verified this to be the culprit cc @rleshchinskiy

[emillon]

it's interesting because the bug does not trigger with just "mkdir node_modules/@x/y; touch node_module/@x/y/z.txt"

[ejgallego]

Indeed, I wonder if my npm is doing some hardlinking on its own? Just in case npm --version = 9.6.7

[ejgallego]

Nothing weird on that side, stat reports entries in path are regular dirs and files.

[emillon]

Yes, I managed to repro with npm too.

stat reports entries in path are regular dirs and files.

To clarify, hardlinks are regular files; did you check the link count too?

[ejgallego]

They have link count larger than 1, I see:

find ~ -samefile _build/default/node_modules/@octokit/auth-token/LICENSE 
/home/egallego/.cache/dune/db/files/v4/dd/dd65cb7fe001574466341c3a0618de18
/home/egallego/tmp/dune-bugs/dune-promote/_build/default/node_modules/@octokit/core/LICENSE
/home/egallego/tmp/dune-bugs/dune-promote/_build/default/node_modules/@octokit/auth-token/LICENSE
/home/egallego/tmp/dune-bugs/dune-promote/_build/default/node_modules/@octokit/request-error/LICENSE

[emillon]

FTR, #9873 points to the same commit, but it's a bit different, it's when a symlink points to a dir.

[ejgallego]

Indeed, symlinks were an obvious suspect, but I can't see any involved here (so far)

[emillon]

one thing I haven't checked is if the issue happens when no directory starts with @. maybe that part is just a red herring?

[ejgallego]

I have checked, if the @ symbol is not in the path, promotion seems to work fine. So it seems related.

[ejgallego]

I've tracked this to the part of https://github.com/ocaml/dune/commit/b1c339b868e00fd5a1a3e72db90e57b4f3285208 that avoids adding "empty" directories:

https://github.com/ocaml/dune/blob/4f86203f77eb7ead7a07286b7728cbf42a386990/src/dune_targets/dune_targets.ml#L246-L248

I am trying to understand what is wrong here.