ocaml-interop

ocaml-interop copied to clipboard

Hi,

I have been having this issue with linking when I was trying to connect the rust library tectonic to ocaml. The library itself compiles fine--as a matter of fact, if I make a standalone rust executable by turning my lib.rs into a main.rs and giving it a minimal fn main() it builds and runs just fine. The issue only happens when I use dune build to build and link ocaml code to my rust code, and it only happens with this specific library, which made its origin a bit mysterious to me. I tried the exact same thing with the library xlsxwriter (which itself is a wrapper around a C library) and it worked like a charm.

The following is one of the linker commands when I run cargo manually (I am exposing both tectonic and xlsxwriter in my lib.rs here):

INFO rustc_codegen_ssa::back::link "cc" "-arch" "arm64" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.0.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.1.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.10.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.11.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.12.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.13.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.14.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.15.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.2.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.3.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.4.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.5.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.6.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.7.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.8.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.build_script_build.15c72c9c-cgu.9.rcgu.o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e.2axekepflnmpc7or.rcgu.o" "-L" "/Users/maclee/Documents/code/rust_bind/target/debug/deps" "-L" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libcc-df41dbe0da785893.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libbindgen-f68437faf5bcabd9.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libwhich-462fb0bd39da582b.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libeither-2e9139c1752ed703.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libshlex-b7fa30b4f8ee236b.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/librustc_hash-1e4044306b864949.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libregex-ebef27c21fd2b8b5.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libaho_corasick-f08b89d1f4585ad5.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libregex_syntax-a8dfe5835e2ac413.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/liblazycell-f7fee489d341656b.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libpeeking_take_while-0ec273cfbf41a962.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libcexpr-b677cb26cedc5008.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libnom-0f44133db0541cc3.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libmemchr-bb958c0e1760d90d.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libclang_sys-3f5ae5102e078103.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/liblibloading-3edd6d7c1e9223b6.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/liblibc-da3d1458e038c6a6.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libglob-ff6bddb9aacb0f81.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/liblog-b65cd57b3b1fe36f.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libcfg_if-d041975b7a578335.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libquote-d1953ea7ee870d28.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libproc_macro2-aabbcb08dc712626.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libunicode_ident-9f71216e14355244.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libproc_macro-84aed0d924f64269.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/liblazy_static-df4e1b8409a9611f.rlib" "/Users/maclee/Documents/code/rust_bind/target/debug/deps/libbitflags-3f7c258c603d2f0d.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libstd-b1513f74602d5fe8.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-c098d20a34c99cef.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libobject-69aebafcf7def0e5.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-bae41e38883b8b61.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-0333bfd34f30f2ac.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libgimli-bc9f4be5546c9aab.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-0d071e3ddea775f7.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-40171c795bc78e11.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-40c757a0b8f5cfef.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-eb17e94441a4ca33.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libadler-fe075f5f8e3d37e1.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-7af5d327fb58cdea.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libunwind-4c91a3fe7821d5ad.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-b366c08ae1e3a3f9.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/liblibc-fc3b8320109cbed1.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/liballoc-1fe6253ea9aba3bb.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-d6b132488ab06c75.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libcore-d1759f0edc4450e0.rlib" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-294c1f759e58027d.rlib" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/opt/local/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/maclee/Documents/code/rust_bind/target/debug/build/libxlsxwriter-sys-887b8c7bddef8f3e/build_script_build-887b8c7bddef8f3e" "-Wl,-dead_strip" "-nodefaultlibs" "-Wl"

The following is the final command dune issued before it hit an error:

$ (cd _build/4.14.0 && /Users/maclee/.opam/4.14.0/bin/ocamlopt.opt -w -40 -g -o src/Main.exe /Users/maclee/.opam/4.14.0/lib/either/either.cmxa /Users/maclee/.opam/4.14.0/lib/containers/monomorphic/containers_monomorphic.cmxa /Users/maclee/.opam/4.14.0/lib/containers/containers.cmxa -I /Users/maclee/.opam/4.14.0/lib/ocaml /Users/maclee/.opam/4.14.0/lib/csv/csv.cmxa src/.Main.eobjs/native/dune__exe.cmx src/.Main.eobjs/native/dune__exe__Common.cmx src/.Main.eobjs/native/dune__exe__Data.cmx src/.Main.eobjs/native/dune__exe__Rosters.cmx src/.Main.eobjs/native/dune__exe__CodeGen.cmx src/.Main.eobjs/native/dune__exe__Ext.cmx src/.Main.eobjs/native/dune__exe__Main.cmx -cclib src/liblab_tools.a)

The first few lines of the >1000 lines of linker output:

Undefined symbols for architecture arm64:
  "_AuthorizationCreate", referenced from:
      security_framework::authorization::Authorization::new::h5048f5f92376623a in liblab_tools.a(security_framework-b4f2f0756325591c.security_framework.c6f64a72-cgu.6.rcgu.o)
  "_AuthorizationCreateFromExternalForm", referenced from:
      _$LT$security_framework..authorization..Authorization$u20$as$u20$core..convert..TryFrom$LT$security_framework_sys..authorization..AuthorizationExternalForm$GT$$GT$::try_from::h9f90c28d0e010fc6 in liblab_tools.a(security_framework-b4f2f0756325591c.security_framework.c6f64a72-cgu.6.rcgu.o)
  "_AuthorizationExecuteWithPrivileges", referenced from:
      security_framework::authorization::Authorization::execute_with_privileges_internal::h7b98a15fd6c55680 in liblab_tools.a(security_framework-b4f2f0756325591c.security_framework.c6f64a72-cgu.6.rcgu.o)
  "_AuthorizationMakeExternalForm", referenced from:
      security_framework::authorization::Authorization::make_external_form::h6cd0f832a31892a8 in liblab_tools.a(security_framework-b4f2f0756325591c.security_framework.c6f64a72-cgu.6.rcgu.o)
  "_CFArrayAppendValue", referenced from:
      _loadAATfont in liblab_tools.a(xetex-macos.o)
      __hb_coretext_shape in liblab_tools.a(harfbuzz.o)

My machine runs macOS on M1 Pro. I also tried this on an x86 Linux machine and a similar result happened (obviously with a slightly different linker output).

Any help would be appreciated.

Hello @macthecadillac. Is this a macOS program? from a quick search what I found is that what is missing is that you need to link the Security.framework framework (how that is done, I am not sure, not very familiar with the apple-specific stuff).

I intend for it to eventually work on macOS, Linux and Windows. I had a hunch that was the problem but I couldn’t find a way to change how dune calls the linker when it comes to “foreign archives” after spending hours looking for it/playing with it yesterday. A similar problem exists on Linux (symbol not found and all that but the output is obviously a bit different) so I don’t think it’s macOS specific problem. If there’s no other way I can obviously split this into a separate rust program and have my ocaml code communicate with it through stdin or something but I prefer having everything in a single binary.On Nov 21, 2022, at 10:06 AM, Bruno Deferrari @.***> wrote: Hello @macthecadillac. Is this a macOS program? from a quick search what I found is that what is missing is that you need to link the Security.framework framework (how that is done, I am not sure, not very familiar with the apple-specific stuff).

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

I might have found a way to completely bypass this problem (mostly by cherry-picking which component crate of tectonic I want to include in my build and circumventing all the network related stuff included by default), but I think it would be helpful if we can get to the bottom of this regardless.