Add MSVC libNAME.lib and dllNAME.dll to stub libraries

[jonahbeckford]

Expected Behavior

This is a continuation of https://github.com/ocaml/dune/pull/6919 which is still unresolved. Instead of trying to fix it myself, better to leave a bug report.

I will use xdg.opam as a simple example.

Here is what should be inside xdg.cma (adapted from ocamlobjinfo/vscode):

File y:\...\.ci\o\dkml\lib\xdg\xdg.cma
Force custom: no
Extra C object files: "-Ly:\...\.ci\o\dkml\lib\xdg" libxdg_stubs.lib shell32.lib ole32.lib uuid.lib
Extra C options:
Extra dynamically-loaded libraries: dllxdg_stubs.dll
Unit name: Xdg
Interfaces imported:
	a9f7c36e5a6e2f4d7151e74af69de9ce	Xdg
	7726a14ce36f5c6c3abb04975cec748b	Stdlib__Sys
	3bcff250dd30a0e620fa7546882aaadf	Stdlib__Filename
	6d7bf11af14ea68354925f3a37387930	Stdlib
	8f8f634558798ee408df3c50a5539b15	CamlinternalFormatBasics
Required globals:
	Stdlib__Filename
	Stdlib__Sys
Uses unsafe features: YES
Primitives declared in this module:
	dune_xdg__get_known_folder_path
Force link: no

Actual Behavior

Notice the incorrect -lxdg_stubs which can't work with MSVC. And the lack of -L so the compiler knows where to look for it.

File y:\...\.ci\o\dkml\lib\xdg\xdg.cma
Force custom: no
Extra C object files: -lxdg_stubs shell32.lib ole32.lib uuid.lib
Extra C options:
Extra dynamically-loaded libraries: -lxdg_stubs
Unit name: Xdg
Interfaces imported:
	a9f7c36e5a6e2f4d7151e74af69de9ce	Xdg
	7726a14ce36f5c6c3abb04975cec748b	Stdlib__Sys
	3bcff250dd30a0e620fa7546882aaadf	Stdlib__Filename
	6d7bf11af14ea68354925f3a37387930	Stdlib
	8f8f634558798ee408df3c50a5539b15	CamlinternalFormatBasics
Required globals:
	Stdlib__Filename
	Stdlib__Sys
Uses unsafe features: YES
Primitives declared in this module:
	dune_xdg__get_known_folder_path
Force link: no

Reproduction (Failed first attempt)

Build xdg.cma:

$ del -force -recurse .\_boot\ ; ocaml boot/bootstrap.ml
$ _boot\dune.exe build .\_build\install\default\lib\xdg\xdg.cma --verbose
...
Command [2] exited with code 2:
$ (cd _build/default && C:\Users\beckf\AppData\Local\Programs\DKMLNA~1\usr\bin\ocamlc.exe -w @[email protected]@30..39@[email protected]@[email protected]@67@69-40 -strict-sequence -strict-formats -short-paths -keep-locs -alert -unstable -g -bin-annot -I otherlibs/xdg/gen_flags/.gen_flags.eobjs/byte -no-alias-deps -opaque -o otherlibs/xdg/gen_flags/.gen_flags.eobjs/byte/dune__exe__Gen_flags.cmi -c -intf otherlibs/xdg/gen_flags/gen_flags.mli)
>> Fatal error: Invalid value for the environment variable BUILD_PATH_PREFIX_MAP: invalid key/value pair "C", no '=' separator
Fatal error: exception Misc.Fatal_error

That was very weird, but this is a known issue: https://github.com/ocaml/dune/pull/7741#issuecomment-1555087662

Reproduction (Ok)

$ opam exec --switch playground -- dune --version
3.12.1

$ opam exec --switch playground -- dune build bin/main.exe

$ opam exec --switch playground -- .\_build\default\bin\main.exe build .\_build\install\default\lib\xdg\xdg.cma

Specifications

• Version of dune (output of dune --version): 3.12.1
• Version of ocaml (output of ocamlc --version): 4.14.0
• Operating system (distribution and version): Windows

[rgrinberg]

cc @nojb

[nojb]

A few questions just to try to understand the issue better:

• This only affects bytecode, is that right?
• I believe Dune outsources the actual building of the mixed OCaml/C libraries to ocamlmklib. Is the problem reported here something that should be addressed in ocamlmklib itself, or rather is it a Dune issue?

[jonahbeckford]

This only affects bytecode, is that right?

It affects bytecode. I don't know what else it affects.

I believe Dune outsources the actual building of the mixed OCaml/C libraries to ocamlmklib. Is the problem reported here something that should be addressed in ocamlmklib itself, or rather is it a Dune issue?

I don't think ocamlmklib is the problem for two reasons:

1. Those ocamlmklib docs are mostly consistent that .dll and .lib should be supplied on Windows. The only exception is "Support libraries for the C part (-llib)" but then it saves itself later by saying "This example is on a Unix system. The exact command lines may be different on other systems."
2. More important, ocamlc -config-var bytecomp_c_libraries has established the "standard" OCaml API that .lib is to be used on Windows: advapi32.lib ws2_32.lib version.lib. We don't say -ladvapi32 -lws2_32 -lversion, and we should be able to feed the output of one OCaml command into another without doing a .lib to -l translation.

There is an argument for ocamlmklib. Of course, ocamlmklib can hide the problem. And the precedent has already been set that ocamlmklib is a Linux-conventions tool ... it outputs .so files even on macOS, with all the ripple effects that decision had.

But my two cents: widening the standard OCaml API to expect -l on Windows will have unforeseen consequences. (Also important: changing ocamlmklib means everyone will have to wait for and use OCaml 5.n)

IMHO, it is better handled by the build systems (ex. dune) to supply ocamlmklib the .lib and .dll as documented.