Halide icon indicating copy to clipboard operation
Halide copied to clipboard

Halide_BUNDLE_LLVM doesn't work with universal builds on OSX

Open derek-gerstmann opened this issue 1 year ago • 6 comments

It appears that bundle_static() in dependencies\llvm\CMakeLists.txt:140 is only injecting the arm64 libs from a universal build of LLVM, when Halide is also configured as a universal build (via CMAKE_OSX_ARCHITECTURES="x86_64;arm64").

The LLVM x86_64 libs are not getting bundled, resulting in missing symbols that show up when linking against libHalide.a:

Undefined symbols for architecture x86_64:
"_LLVMInitializeAArch64AsmParser", 
"..."

I'm not entirely sure why only the arm64 libs get bundled. Is the llvm_map_components_to_libnames provided with the LLVM project skipping them?

derek-gerstmann avatar Mar 04 '24 23:03 derek-gerstmann

@alexreinking Any thoughts?

derek-gerstmann avatar Mar 04 '24 23:03 derek-gerstmann

We also need the generator helpers to produce fat binaries when CMAKE_OSX_ARCHITECTURES is set. There is a small complication with specifying target features for multiple, incompatible architectures (e.g. avx2 and arm_dot_prod)

alexreinking avatar Aug 03 '24 23:08 alexreinking

Currently there's no way to have a single Generator run produce multiple architectures (multitarget requires the same base runtime feature selection). How are these produced on OSX? e.g. can you produce a .a file that is "fat"?

steven-johnson avatar Aug 05 '24 15:08 steven-johnson

We will need to run the generator once for each platform in CMAKE_OSX_ARCHITECTURES. For example, if that variable is set to x86_64;arm64 then we will need to run the generator for x86-64-osx and arm-64-osx. This will produce two .a files, which we can merge into a "fat" one using lipo (the irony!).


Brainstorming: there is an issue with specifying features in the fat binaries case. We could add platform-specific FEATURES arguments, like so:

add_halide_library(
  myFilter FROM myGen
  FEATURES user_context
  FEATURES[arm-64] "arm_dot_prod-arm_fp16" "arm_fp16"
  FEATURES[x86] "avx2"
)

The [] brackets contain a prefix of the arch-bits-os platform triple to match against. The longest match wins. Providing multiple sets of features implies multi-target. Unqualified FEATURES applies everywhere. This gives us a clear way to specify what to do when we need to generate multiple times for the sake of fat binaries.

alexreinking avatar Aug 05 '24 16:08 alexreinking

I think allowing a prefix is probably more general than necessary and harder to implement and document... I'd vote for just a triple with all of arch-bits-os, that should allow what we need with only occasional extra verbosity

steven-johnson avatar Aug 06 '24 15:08 steven-johnson

Sure - that leaves the door open to implementing prefixes or another policy if users ask. (Invalid syntax becomes valid)

alexreinking avatar Aug 06 '24 15:08 alexreinking

Fixed by #8390

alexreinking avatar Sep 10 '24 17:09 alexreinking