`js_of_ocaml` and `wasm_of_ocaml` support for parametric libraries

[shonfeder]

[hhugo]

Can someone explain what's the issue about ? Is there any work needed on the jsoo/wasmoo side ? or is it just on the dune side ?

[art-w]

Yes sure! This is about an experimental extension called "parameterised libraries" that is being tested in OxCaml to support the functorisation of whole libraries (~ it's a better version of the "virtual module" linking trick that dune provides, and is not as invasive as manual functorizing). I believe some work will be needed in jsoo to support separate compilation.

Currently you need to install the OxCaml compiler to play with it:

$ ocamlc -config | grep param
parameterised_modules: true

There's an example test in this PR to show that it works in dune with jsoo/wasmoo whole program compilation. However the separate compilation with jsoo currently triggers a bytecode parsing error, as seen if we call the compiler manually.

To do so, we need a bit of setup. A "parameter" is a single mli interface:

$ echo 'val foo : string' > param.mli
$ ocamlc -as-parameter param.mli

A module can then be parameterised, which allows its code to refer to the parameter by its signature name (even though we don't have an implementation at this point, i.e. it's a bit like wrapping the code in functor (Param:Param) -> ...):

$ echo 'let lib () = "lib(" ^ Param.foo ^ ")"' > lib.ml
$ ocamlc -parameter Param lib.ml

A parameter can be implemented:

$ echo 'let foo = "impl"' > impl.ml
$ ocamlc -as-argument-for Param impl.ml

And then we can instantiate the library with that implementation (note the new dash - syntax in the resulting filename):

$ ocamlc -instantiate lib.cmo impl.cmo -o lib-Impl.cmo

(When compared to virtual modules, the "parameterised library" feature adds the possibility to instantiate the same lib with different arguments, instead of being restricted to one. When compared with standard functors, it handles the instantiation of multiple modules in a transparent way; also the parameters are identified by their name instead of their position in the application, which makes it easy to depend on other parameterised libraries.)

Finally, an executable can access the instantiated library with some special syntax (which resembles functor application, but the special attribute triggers the parameter nominal-argument magic):

$ cat > bin.ml <<EOF
  module X = Lib(Param)(Impl) [@jane.non_erasable.instances]
  let () = print_endline (X.lib ())
EOF
$ ocamlc impl.cmo lib.cmo lib-Impl.cmo bin.ml -o a.out

Now the whole program compilation with jsoo already works fine:

$ js_of_ocaml a.out -o a_out.js

But the compilation of the intermediate lib-Impl.cmo does not:

$ js_of_ocaml lib-Impl.cmo
js_of_ocaml: You found a bug. Please report it at https://github.com/ocsigen/js_of_ocaml/issues :
Error: File "compiler/lib/parse_bytecode.ml", line 1386, characters 8-14: Assertion failed
Raised by primitive operation at Stdlib__Sys.(partial) in file "sys.ml.in" (inlined), line 173, characters 0-82
Called from Dune__exe__Js_of_ocaml in file "compiler/bin-js_of_ocaml/js_of_ocaml.ml", lines 46-71, characters 4-821

(I suspect it's tripping on something simple, but I don't know how the instantiation is represented in the bytecode)

[hhugo]

The failure comes from the fact that at the bytecode level, lib-Impl.cmo wants to "register" the compilation unit Lib while depending on another compilation unit Lib. It contains relocation information for both Reloc_getcompunit "Lib" (as it seems to depend on Lib) and Reloc_setcompunit "Lib". From a distance, it seems it could be an issue with the bytecode gen in oxcaml.

[hhugo]

The issue seems to be in the jsoo support for oxcaml. see https://github.com/ocsigen/js_of_ocaml/pull/2105#discussion_r2597995219.