`dune install` with an out-of-workspace build directory produces absolute paths in dune-package files

[purplearmadillo77]

Expected Behavior

After following the below reproduction steps, I would expect ~/fake_did/lib/stdlib-shims/dune-package to contain

(library
 (name stdlib-shims)
 (kind normal)
 (archives (byte stdlib_shims.cma) (native stdlib_shims.cmxa))
 (plugins (byte stdlib_shims.cma) (native stdlib_shims.cmxs))
 (modes byte native)
 (modules (unwrapped)))

Actual Behavior

After following the below reproduction steps, if you view ~/fake_did/lib/stdlib-shims/dune-package, you will see the following stanza:

(library
 (name stdlib-shims)
 (kind normal)
 (archives
  (byte
   /Users/[username]/fake_dbd/install/default/lib/stdlib-shims/stdlib_shims.cma)
  (native
   /Users/[username]/fake_dbd/install/default/lib/stdlib-shims/stdlib_shims.cmxa))
 (plugins
  (byte
   /Users/[username]/fake_dbd/install/default/lib/stdlib-shims/stdlib_shims.cma)
  (native
   /Users/[username]/fake_dbd/install/default/lib/stdlib-shims/stdlib_shims.cmxs))
 (modes byte native)
 (modules (unwrapped)))

If I try to use this dune-package file, dune will search for the cma/cmxa files on those absolute paths. This means if I were to distribute this file (e.g. building packages for package managers) then end-user computers will look for the libraries on nonexistent paths, rendering stdlib-shims somewhat unusable.

Is there a setting to ask dune to not output such absolute paths? It feels like the build location for a package should not affect its usage behavior.

Reproduction

• PR with a reproducing test: TODO

Steps to reproduce:

1. Create a build folder ~/fake_dbd
2. Create an installation folder ~/fake_did
3. Create a folder ~/test_dune and within it:
4. git clone https://github.com/ocaml/stdlib-shims.git
5. Build stdlib-shims with DUNE_BUILD_DIR=~/fake_dbd dune build -p stdlib-shims --verbose
6. Install stdlib-shims with DUNE_BUILD_DIR=~/fake_dbd DUNE_INSTALL_PREFIX=~/fake_did dune install --verbose

Specifications

• Version of dune (output of dune --version): 3.18
• Version of ocaml (output of ocamlc --version): 4.12.0
• Operating system (distribution and version): macOS Monterey 12.7.6

[nojb]

I think this request makes sense. Having said that, note that the OCaml toolchain is not currently relocatable so the compilation artifacts themselves may not be relocatable. But as mentioned, I think emitting relative paths in dune-package and interpreting them relative to the location of dune-package sounds reasonable. @rgrinberg: do you confirm?

[nojb]

On further investigation, this looks like a bug because if you use the usual build directory (ie you do not set DUNE_BUILD_DIR), then Dune uses relative file names in dune-package.

[nojb]

On further investigation, this looks like a bug because if you use the usual build directory (ie you do not set DUNE_BUILD_DIR), then Dune uses relative file names in dune-package.

Indeed, (the following comment is meant for Dune maintainers), the issue is that we call https://github.com/ocaml/dune/blob/6bc42be39a337c92e64c41b3e49c24356ae45055/src/dune_lang/path.ml#L7-L10 to generate the paths that go in dune-package. But Path.reach ~from t returns Path.to_string t whenever t is an external path (which is the case when using an out-of-workspace build directory).

A possible approach would be to handle the case when t is a direct descendent of ~from, even for external paths. Not sure if this can break anything.

[rgrinberg]

Emitting relative paths in the dune-package should be preferred. We should produce relocatable when possible. Customizing the build directory is not a well tested feature, so I'm not surprised there's bugs like this.