cmxs should not be installed in libexec

[bobot]

The fix for setting the exec bit of .cmxs (issue #4148) done in #4149, was wrong because seemingly a misunderstanding of the meaning of libexec. The destination of Libexec can be different from Lib.

CC @olafhering

[bobot]

We have two meanings:

• opam install files where libexec is the same directory than lib but with the bit set.
• Distribution layout, where libexec is different from lib and usually used for private executables that should not appear in /usr/bin

[bobot]

So internally I propose to have:

	internally named	opam install file	dune install
lib without exec bit (used for .cmo, .a, ...)	Section.Lib	lib	lib without exec bit
lib with exec bit (used for .cmxs)	Section.Lib_with_exec_bit	libexec	lib with exec bit
libexec	Section.Libexec	libexec	libexec with exec bit

Internally we can also have a bigger change by adding for each file the install mode to use for the file. Still we also need to change the install stanza.

[psafont]

Since xapi-project installs binaries to libexec, I tried to take a stab at renaming the install stanza before adding the new libexec destination.

While the change itself is not complex it breaks quite a bit of test dependencies (ppxlib, ppx_expect, mdx, dune-site), which means tests cannot be promoted willy-nilly, and it makes the change quite complex.

What would be the recommended way to push forward? I thought about installing this new version of dune to the opam switch and update all the dependencies that break until the tests are reasonable. Would this work?

In any case it seems quite a painful change to do, so I'm not too convinced the community will buy into the change, maybe the idea to handle executable bits in another way makes sense if there needs to be a breaking change anyway.

[rgrinberg]

While the change itself is not complex it breaks quite a bit of test dependencies (ppxlib, ppx_expect, mdx, dune-site), which means tests cannot be promoted willy-nilly, and it makes the change quite complex.

How does it break them?

What would be the recommended way to push forward? I thought about installing this new version of dune to the opam switch and update all the dependencies that break until the tests are reasonable. Would this work?

The conservative way to do this would be to guard the change behind the version of the dune language that is set in the dune-project file.

[nojb]

Sorry, but can someone state precisely what is being done currently by Dune and what user-visible changes are being proposed?

[psafont]

Sorry about the late update, it got lost in a sea of notifications

How does it break them? changing the name of the stanza makes dune unable to recognize the new location

The conservative way to do this would be to guard the change behind the version of the dune language that is set in the dune-project file.

I'll try this, thanks

Sorry, but can someone state precisely what is being done currently by Dune and what user-visible changes are being proposed?

User packages ask libraries to be installed in the 'libexec' section. Currently this means that these are installed in the 'lib' location, but with the execution bit set. This is confusing because libexec is a FHS-defined location, different from the 'lib' one.

The plan is to rename the dune-define libexec section, this will purposely fail when dune runs. Example from my PoC:

./_boot/dune.exe build @install
File "/home/paus/.local/share/opam/dune/lib/lwt/dune-package", line 4, characters 19-26:
4 | (sections (lib .) (libexec .) (doc ../../doc/lwt) (stublibs ../stublibs))
                       ^^^^^^^
Error: Unknown value libexec

The error should be made much better to indicate what's replacement, documentation needs to mention this, etc. Maybe even introduce a transition period where it's deprecated and warnings are shown before breaking it.

After the breaking change is done and made obvious to users, a libexec section is introduced, that actually follows the FHS spec

[nojb]

Thank you for the explanation.

As a general rule, Dune observes a strict backwards-compatibility policy, so straightaway renaming or changing the meaning of existing Dune install section names is out of the question.

Taking a step back, I see two distinct issues here:

1. There is no Dune install section for the the "FHS libexec" (the libexec/ directory).

2. The existing libexec install section means "install in lib/ with executable bit" and this may be confusing to some people.

I would concentrate first on fixing (1): introduce a new Dune install section for the "FHS libexec". This can be done without breaking anything. Addressing the second point (changing the meaning of the existing libexec install section to mean the "FHS libexec") needs to be opt-in until the next major version of Dune because it will silently break people's install stanzas.

[psafont]

Sure, I'll add the new section. I'll use the name libexec_fhs for now, unless somebody has a better name for it :)

The plan for the switch-up can be thought up later