Linker errors when building on Windows

[RossSmyth]

Building Roc on Windows is not very fun at the moment. Thank you to @folkertdev for all the help on Zulip. Currently, building Roc as-is on Windows just does not work. All of the errors encountered are linker errors. The errors are the following:

  = note: lld: error: undefined symbol: roc_alloc
          >>> referenced by C:\Users\Ross\Documents\roc\crates\roc_std\src\roc_list.rs:70
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(roc_std::roc_list::RocList$LT$T$GT$::elems_with_capacity::h2483f6d70c7c27a7)
          >>> referenced by C:\Users\crs20\Documents\roc\crates\roc_std\src\lib.rs:51
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(roc_std::roc_alloc_refcounted_help::h08260161a777fba6)

          lld: error: undefined symbol: roc_realloc
          >>> referenced by C:\Users\Ross\Documents\roc\crates\roc_std\src\roc_list.rs:174
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(roc_std::roc_list::RocList$LT$T$GT$::reserve::hb395b2048ff799d4)
          >>> referenced by C:\Users\crs20\Documents\roc\crates\roc_std\src\roc_list.rs:270
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(roc_std::roc_list::RocList$LT$T$GT$::extend_from_slice::h7698bf583e077d5e)

          lld: error: undefined symbol: roc_dealloc
          >>> referenced by C:\Users\Ross\Documents\roc\crates\roc_std\src\roc_list.rs:220
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(roc_std::roc_list::RocList$LT$T$GT$::reserve::hb395b2048ff799d4)
          >>> referenced by C:\Users\Ross\Documents\roc\crates\roc_std\src\roc_list.rs:464
          >>>               libroc_std-c05e5e25b8ffcb50.rlib(roc_std-c05e5e25b8ffcb50.1rxfp4xdyhetnw4k.rcgu.o):(_$LT$roc_std..roc_list..RocList$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::h00aee7c542f8a77b)

Note that have set the linker to be LLD (RUSTFLAGS = "-Clinker=lld"), as that is my preferred linker, but that same error occurs with MSVC's LINK.exe

Following the instructions in BUILDING_FROM_SOURCE.md got me partially there. Using the build LLVM in the 7z file, and setting the LLVM_SYS_130_PREFIX environment variable did work. Then run: cargo +1.61.0-msvc build And you will see the error above (or the equivalent LINK.exe error).

The error is occurring because it is trying to link in the roc_alloc, roc_realloc, and roc_dealloc symbols in the following crates:

• roc_load
• roc_cli
• roc_docs
• roc_docs_cli
• repl_test

The first one hit is roc_load, so that was foxused on at first. Setting the ROC_SKIP_SUBS_CACHE environment variable did not let it build. folkertdev created a branch that removed a lot of things in this crate to try and work on Windows here. That commit as-is still did not work, and my solution was to just delete the "build.rs" file completely. Then the other crates had errors, and folkertdev suggested to remove them from the top-level cargo.toml. After this I was about to build the project. But this removes all the CLI executables so I'm not sure how useful it is.

The steps to build are:

1. rm .\crates\compiler\load\build.rs
2. Apply this patch

diff --git a/Cargo.toml b/Cargo.toml
index 014589c6a..045b6c8df 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -37,20 +37,16 @@ members = [
     "crates/glue",
     "crates/editor",
     "crates/ast",
-    "crates/cli",
     "crates/code_markup",
     "crates/highlight",
     "crates/error_macros",
     "crates/reporting",
     "crates/repl_cli",
     "crates/repl_eval",
-    "crates/repl_test",
     "crates/repl_wasm",
     "crates/repl_expect",
     "crates/test_utils",
     "crates/utils",
-    "crates/docs",
-    "crates/docs_cli",
     "crates/linker",
     "crates/wasi-libc-sys",
 ]

This only allows for cargo +1.61.0-msvc build to run successfully. Running any executables still does not work, and cargo +1.61.0-msvc test has the same linker failures, except you can't just remove things until it works because for any useful tests to run, they depend upon those symbols existing.

Here is a branch with the above changes: https://github.com/RossSmyth/roc/commit/e06a8efcf8f058ee3a8052f461cdb639c0fb47d6

[RossSmyth]

Looking it I think that these symbols are supposed to come from the host language and managed by the Roc compiler.

[RossSmyth]

One thought that I have is that it is because of transitive dependencies.

There is a note here: https://github.com/roc-lang/roc/blob/603f6b6613d7d8380a5690700918a3e64b5ae013/Cargo.toml#L65-L66 About the issues above, as most of them original from here: https://github.com/roc-lang/roc/blob/0fab77af99f970245f04da85a6fd645caa7063cb/crates/roc_std/src/roc_list.rs#L16 Or elsewhere in roc_std is used, it will still try to pull it in, as this will not stop cargo from attempting to build it if it is a transitive dependency.

[RossSmyth]

And to make sure I'm not insane, I can compile it fine in WSL2.

[folkertdev]

we concluded earlier that the problem is insufficient dead-code elimination. The roc_alloc function and its friends are not actually used by the compiler code. On *nix systems that means the roc_alloc function is DCE'd and it's not a problem for the linker. On windows this is clearly not the case.

That may mean we need a feature flag that either gives just the types in roc_std, or both the types and implementations

[folkertdev]

to get the cli to compile you now need to disable repl_expect and add some definitions to cli/src/lib.rs to satisfy the linker

use core::ffi::c_void;

/// # Safety
/// The Roc application needs this.
#[no_mangle]
pub unsafe fn roc_alloc(size: usize, _alignment: u32) -> *mut c_void {
    libc::malloc(size)
}

/// # Safety
/// The Roc application needs this.
#[no_mangle]
pub unsafe fn roc_memcpy(dest: *mut c_void, src: *const c_void, bytes: usize) -> *mut c_void {
    libc::memcpy(dest, src, bytes)
}

/// # Safety
/// The Roc application needs this.
#[no_mangle]
pub unsafe fn roc_realloc(
    c_ptr: *mut c_void,
    new_size: usize,
    _old_size: usize,
    _alignment: u32,
) -> *mut c_void {
    libc::realloc(c_ptr, new_size)
}

/// # Safety
/// The Roc application needs this.
#[no_mangle]
pub unsafe fn roc_dealloc(c_ptr: *mut c_void, _alignment: u32) {
    libc::free(c_ptr)
}

[folkertdev]

fixed by #3901

cargo build works on that commit. It's not in CI yet so some things may break again. We'll try to at least get a cargo check CI job set up soon. Also testing probably still fails because of missing symbols in some places.

in any case, we're working on it, and the workarounds in this issue should no longer be needed.