Package build rules can't run ocamlfind's `ocaml` shell script on Windows

[gridbugs]

Packages that depend on ocamlfind and run the command ocaml run a wrapper shell script from the ocamlfind package rather than the ocaml executable from the compiler toolchain. On Windows, scripts cannot be run as executable files as they can on Unix, and we see the error:

> dune build
Error: CreateProcess(): Exec format error
-> required by _build/_private/default/.pkg/topkg/target

The ocamlfind package contains a shell script called ocaml which wraps the ocaml executable from the compiler toolchain and passes it some extra arguments. The script is:

#!/bin/sh

BINDIR=$(dirname "$(command -v ocamlc)")
"$BINDIR/ocaml" -I "$OCAML_TOPLEVEL_PATH" "$@"

Packages that build with topkg usually run this wrapper script to invoke their build scripts (which are small ocaml programs). For example here is uutf's build command:

build: [
  "ocaml"
  "pkg/pkg.ml"
  "build"
  "--dev-pkg"
  "%{dev}%"
  "--with-cmdliner"
  "%{cmdliner:installed}%"
]

Dune package management can't handle this type of build command on Windows as it invokes a script (the wrapper shell script ocaml) as an executable, which is not allowed on Windows. The workaround is to change the lockfile manually for packages with build commands like this to explicitly launch sh and pass the script as an argument. For example, here is the original lockfile for uutf in a project which depends on it:

(build
 (run
  ocaml
  pkg/pkg.ml
  build
  --dev-pkg
  %{pkg-self:dev}
  --with-cmdliner
  %{pkg:cmdliner:installed}))

After applying the workaround it looks like:

(build
 (run
  sh
  %{pkg:ocamlfind:bin}/ocaml
  pkg/pkg.ml
  build
  --dev-pkg
  %{pkg-self:dev}
  --with-cmdliner
  %{pkg:cmdliner:installed}))

Repro case: https://github.com/gridbugs/dune-windows-repros/tree/main/repro5

See also: https://github.com/ocaml/dune/issues/11174 which is about packages that run a configure script as part of their build process which also fails on Windows due to attempting to run a script as an executable file.

[Leonidas-from-XIV]

I don't think changing the lock file is the right direction, as that creates different lock files between the systems while the lock files should just describe what should happen and then the Dune engine should make that work.

How does OPAM solve this on Windows? OPAM files are the same on every platform.

[gridbugs]

I don't think changing the lock file is the right direction, as that creates different lock files between the systems while the lock files should just describe what should happen and then the Dune engine should make that work.

I agree! Changing the lock file is just a workaround to demonstrate the problem and allow other problems to be discovered.

How does OPAM solve this on Windows? OPAM files are the same on every platform.

Here's an explanation from a related issue: https://github.com/ocaml/dune/issues/11174#issuecomment-2762344653

[Leonidas-from-XIV]

Here's an explanation from a related issue: https://github.com/ocaml/dune/issues/11174#issuecomment-2762344653

This is fairly hacky but I have to admit that it is probably as good of a solution as it gets. I remember that Windows had a mechanism to register file extensions to be automatically run with the respective interpreter (.vbs, .js, .py come to mind) but I don't think we can rely on this to execute .sh or files without an extension in the first place as is fairly common on Unix. So emulating a Unix kernel and evaluating the shebang line is probably the best course of action.