stubgen: support cross compilation of generated modules

[mbacarella]

Hi there,

One way of using ctypes stubgen involves building and running this OCaml program:

let () =
  print_endline "#include <mpg123.h>";
  Cstubs_structs.write_c Format.std_formatter
    (module Mpg123_c_type_descriptions.Types)
;;

that outputs a C program like this:

#include <mpg123.h>
#include <stdio.h>
#include <stddef.h>
#include "ctypes_cstubs_internals.h"

int main(void)
{

  puts("include Ctypes");
  puts("let lift x = x");
  puts("open Ctypes_static");
  puts("");
  puts("let rec field : type t a. t typ -> string -> a typ -> (a, t) field =");
  puts("  fun s fname ftype -> match s, fname with");
  puts("  | Struct ({ tag = \"`Id3v2\"} as s'), \"comment\" ->");
  ctypes_printf("    let f = {ftype; fname; foffset = %zu} in \n",
               offsetof(mpg123_id3v2, comment));
  puts("    (s'.fields <- BoxedField f :: s'.fields; f)");
  puts("  | Struct ({ tag = \"`Id3v2\"} as s'), \"genre\" ->");
  ctypes_printf("    let f = {ftype; fname; foffset = %zu} in \n",
               offsetof(mpg123_id3v2, genre));
  puts("    (s'.fields <- BoxedField f :: s'.fields; f)");
  puts("  | Struct ({ tag = \"`Id3v2\"} as s'), \"year\" ->");
  ctypes_printf("    let f = {ftype; fname; foffset = %zu} in \n",
               offsetof(mpg123_id3v2, year));
  puts("    (s'.fields <- BoxedField f :: s'.fields; f)");
  puts("  | Struct ({ tag = \"`Id3v2\"} as s'), \"album\" ->");
  ctypes_printf("    let f = {ftype; fname; foffset = %zu} in \n",
               offsetof(mpg123_id3v2, album));
  puts("    (s'.fields <- BoxedField f :: s'.fields; f)");
  puts("  | Struct ({ tag = \"`Id3v2\"} as s'), \"artist\" ->");
  ctypes_printf("    let f = {ftype; fname; foffset = %zu} in \n",
               offsetof(mpg123_id3v2, artist));
...
  return 0;
}

which, when built and run outputs an ocaml module that should be built into your project to use the C stubs.

This doesn't work for cross compilation. I notice @whitequark did some sweet hack to get ctypes itself to cross compile https://github.com/ocamllabs/ocaml-ctypes/pull/383

Would be nice to do something like this for ctypes users as well. I would like to add this to dune's upcoming native ctypes rules generation support https://github.com/ocaml/dune/pull/3905

Any thoughts on proceeding?

[nojb]

Note that the "hack" that you mention is already in dune's codebase, it is used in configurator, see https://github.com/ocaml/dune/blob/bdfe5291bf2e080d64d8543c41ba55dd281bee22/otherlibs/configurator/src/v1.ml#L531-L537 and the code around it.

[mbacarella]

To be clear I mean hack in the best possible meaning of the term :)

It's true dune does do something like this to discover definitions and values from compiled C code but it is not apparently easily adapted to letting us pluck that substantial OCaml module out of a compiled binary. Also Cstubs_structs.write_c_for_cross_compilation seems like a better home for it.

[yallop]

This would be nice to have, and Cstubs_structs.write_c_for_cross_compilation seems like a reasonable place to put it.

One question is whether to switch the whole process over to use @whitequark's approach or maintain two distinct flows for type stubs generation.