add proper `libb2` dependency to `unison-runtime`

[aryairani]

@aryairani

So there’s something about the structure of unison-runtime.rkt or a dependency that causes libcrypto and libssl to be bundled when we run raco distribute unison-runtime, but libb2 is not bundled in the same way, which is causing headaches. @dolio Do you have any insights into this?

@dolio

Not really. I think the first two are used in 'standard' racket libraries, and maybe the last one is us using the racket FFI? So maybe it doesn't do the same bundling for the latter. Unsure if you can tell it otherwise. We do do the same sort of ffi import for libcrypto (in the same file), but I think we might also depend on racket libraries that use libcrypto, which might be the important factor.

@aryairani

I skimmed the docs and didn’t see any mention; I wondered if you could take a look at the dependencies you think libcrypto or libssl might be coming from to see if there’s a magic pragma we would need to include to trigger the bundling for libb2

I found this: https://github.com/racket/racket/blob/f6fe40dde18004d71ece98a2d22de84e42e90bcf/racket/collects/openssl/libcrypto.rkt#L77-L86 Discord also says:

You probably need to use define-runtime-path to specify the location of libb2

So we probably just need to add that.

[dolio]

I created a branch jit-bundle that adds a define-runtime-path for the libb2 library in the crypto file.

It builds successfully for me. However, it doesn't seem like I can test if it'll work, because raco exe/distribute doesn't bundle any libraries in the generated directory for me. So you'll have to see if it does something on Mac, @aryairani.

[aryairani]

@dolio Ok thanks, I will try it out.

It might be because I called

raco exe --embed-dlls --orig-exe scheme-libs/racket/unison-runtime.rkt

I'm not totally sure what those two flags do, or what OSs they have an effect on, but I'm guessing they're responsible for you not getting any libraries.

Or wait, I just realized you probably meant that on your local machine, when you run it without those flags, you also don't get any libraries bundled....

[dolio]

Yeah, I don't get any libraries in the directory with just raco exe .../unison-runtime.rkt raco distribute ....

I assume that on Linux all the libraries are 'system libraries' or whatever and assumed to be installed separately.

[aryairani]

I just found this callout on the raco distribute page

On Windows and Mac OS, native libraries tend to be included with the output of raco distribute. On Unix platforms, native libraries tend not to be included, so system libraries will be used on the host machine. The difference is whether a Racket installation itself includes bundled native libraries or relies on system-installed libraries. Adding a symbolic link in Racket’s "lib" directory to a system-installed library causes that library to be included with a distribution directory created by raco distribute; see also define-runtime-path.

Where is Racket's "lib" directory though?

The changes in topic/jit-bundle alone didn't seem to make a difference on Mac either. (The "unison-runtime" build artifacts on the CI page for the PR give the raco distribute output for all three platforms.) I realized the release workflow doesn't have libb2 installed during raco exe or raco distribute; however it also didn't complain about missing libb2 so I don't know what define-runtime-path is actually doing, if anything.

[dolio]

I don't know what the Mac install looks like, but I would guess there's a lib subdirectory of it or something. I don't see one on Linux, but that's consistent with them just relying on system libraries.

[aryairani]

Somehow your update on the branch didn't make it into the racket source that's getting built by CI, so I will look into that.