Error solving projects with dependencies in & out of workspace

[ElectreAAS]

I've tried to build several highly depended-upon packages with the new package management features to improve coverage, and here are some results:

---> Building package n°14: package "dune-configurator.3.16.0":
Successfully extracted to /tmp/pkg-coverage-57b851-builder/dune-configurator.3.16.0
Error: Dune does not support packages outside the workspace depending on
packages in the workspace. The package "base" is not in the workspace but it
depends on the package "dune" which is in the workspace.


---> Building package n°24: package "js_of_ocaml.5.8.2":
Successfully extracted to /tmp/pkg-coverage-9f3e89-builder/js_of_ocaml.5.8.2
Error: Dune does not support packages outside the workspace depending on
packages in the workspace. The package "logs" is not in the workspace but it
depends on the package "js_of_ocaml" which is in the workspace.
                      

---> Building package n°25: package "js_of_ocaml-compiler.5.8.2":
Successfully extracted to /tmp/pkg-coverage-f612f2-builder/js_of_ocaml-compiler.5.8.2
Error: Dune does not support packages outside the workspace depending on
packages in the workspace. The package "logs" is not in the workspace but it
depends on the package "js_of_ocaml" which is in the workspace.

All three of these packages seem rather important to me, especially since they have (according to check.ci.ocaml.org) 4340 revdeps each.

[Leonidas-from-XIV]

I believe this might be an issue with our coverage tool (which currently locks all projects in the source tarball). What we need is probably a way to lock including only certain packages. I believe if we could have

$ dune pkg lock -p dune-configurator
$ dune pkg lock -p js_of_ocaml
$ dune pkg lock -p js_of_ocaml-compiler

these would work and (like opam) resolve their dependencies through the opam-repo.

[rgrinberg]

What would such a -p flag be useful for in practice? If there's a dependency coming from the workspace, in the vast majority of cases we'd like to use it.

[Leonidas-from-XIV]

For example, if you have parts of your projects that you don't release. Yojson for example has Yojson-bench as part of its repo (and tarball), but it is not released to OPAM. This works fine at the moment in OPAM because OPAM calls dune build -p yojson so it doesn't pick up the source nor the dependencies of yojson-bench.

[rgrinberg]

That's different and has been covered by https://github.com/ocaml/dune/issues/9688. As a side note, I don't think it's all that useful of a feature. In your example, all you're doing is just avoiding solving for the dependencies of yojson-bench. Not a huge benefit.

This example means something else:

$ dune pkg lock -p js_of_ocaml

Because you aren't just skipping a package in the workspace, but rather replacing it with a package inside the opam repository. This is only useful for making opam releases. I don't see why users would want this in their local builds.

[maiste]

Possible solution mentioned during the Dune dev meeting here: https://github.com/ocaml/dune/issues/8652

[smorimoto]

Any updates?

[maiste]

Any updates?

We are still discussing on the way to solve the problem. Today, it seems to require a lot of changes to fix it, and we want to be sure about the how. This is definitely something important we have in "our backlog" 👍

[smorimoto]

Fantastic!