Mediapipe LLM build fails for Android X86_64

[vraghavulu]

trafficstars

OS Platform and Distribution

Ubuntu 22.04, Android 14

Compiler version

Build failure with CLANG 9.0.0

Programming Language and version

C++, Java, Python 3.10

Installed using virtualenv? pip? Conda?(if python)

No response

MediaPipe version

0.10.15

Bazel version

7.1 & 6.5.0

XCode and Tulsi versions (if iOS)

No response

Android SDK and NDK versions (if android)

Build failure with NDK=22 & SDK=30

Android AAR (if android)

None

OpenCV version (if running on desktop)

No response

Describe the problem

MediaPipe LLM inference task build for Android X86_64 fails with toolchain error

Complete Logs

I started to build MediaPipe FW, sample applications for LLM & non-LLM use-cases for Android x86_64 platforms. The build fails due to not using latest GCC/Clang tool chains (&NDK version). Did few modifications with which build succeeds & run on x86 Android platforms (Android studio emulator, WSA). I am highlighting those changes that were did for the build to succeed.

I want inputs in formalizing these changes to MediaPipe so as to build it for Android X86_64 platforms running on 12th Gen Intel Core Processors & higher.

The topmost commit id that I had is:4f14ecb5794eea8dd46255b64b81f1808dd69d04

Firstly, the following configs in .bazelrc were added to configure the build for X86_64
build:android_x86_64 --config=android
build:android_x86_64 --cpu=x86_64
build:android_x86_64 --fat_apk_cpu=x86_64

build:android_x86 --config=android
build:android_x86 --cpu=x86
build:android_x86 --fat_apk_cpu=x86

After adding above configs, the following builds cmds were used after to build the one or more components:
bazel build -c opt --fat_apk_cpu=x86_64 --strip=never --host_crosstool_top=@bazel_tools//tools/cpp:toolchain mediapipe/tasks/java/com/google/mediapipe/tasks/vision:tasks_vision

bazel build -s -c dbg --strip=never --config=android_x86_64 --host_crosstool_top=@bazel_tools//tools/cpp:toolchain mediapipe/tasks/java/com/google/mediapipe/tasks/genai:libllm_inference_engine_jni.so

bazel build -c opt --config=android_x86_64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu:objectdetectioncpu

Two major build issues observed:
Issue 1. Cannot support flag -mamx-int8 -mavxvnni while building the XNNPACK component. Issue resolution can be done by moving to higher GCC/Clang versions by using NDK25. Errors logs are like this:

ERROR: /media/venkat/local_disk/MP/xnnp/XNNPACK/BUILD.bazel:1894:19: Compiling src/amalgam/gen/avxvnni.c failed: (Exit 1): gcc failed: error executing CppCompile command (from target //:avxvnni_prod_microkernels) /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g -MD -MF ... (remaining 32 arguments skipped)
 Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
gcc: error: unrecognized command line option '-mavxvnni'; did you mean '-mavx512vnni'?

Issue 2. When building LLM tasks (and or libllm_inference_engine_jni.so) build fails with multiple issues
Build cmd: bazel build -s -c dbg --strip=never --config=android_x86_64 --host_crosstool_top=@bazel_tools//tools/cpp:toolchain //mediapipe/tasks/java/com/google/mediapipe/tasks/genai:libllm_inference_engine_jni.so

The most prominent error occurs due to “gcc tool chain” error. Analysis of error leads to the gcc toolchain error in the file ~/.cache/bazel/_bazel_root/b160314324a439d32bbb8d7e564e155e/external/androidndk/cc_toolchain_config.bzl.
Changes done: 
a. Use higher Bazel version (bazel-6.5.0) 
b. Added changes to include configs for android_x86_64 and eventually use android-ndk-r25b (for CLANG toolchain support) which has support for Intel ISA instructions like AVX512AMX, AVXVNNI instructions.
c. Added changes in the file 'third_party/android/android_configure.bzl' to add STARLARK_RULES. 

Can MediaPipe team help improve/formalize these changes?

[kuaashish]

Hi @vraghavulu,

As far as I know, currently, libmediapipe_tasks_vision_jni.so is available for x86, arm64-v8a, and arme-abiv7a, but not for x86_64. This might be the reason for the build failure. Please allow me some time to discuss with the team, and I will provide confirmation on this.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[vraghavulu]

Hi @kuaashish , Do you have any updates regarding this issue?

Is there any plant to move to higher NDK version that can support higher CLANG versions? This will partially resolve the issue of building the advanced X86_64 instructions?

[kuaashish]

Hi @vraghavulu,

At the moment, we do not have any information about this. However, we are marking it as a feature request and sharing it with the team so it will be on our radar.

Thank you!!

[schmidt-sebastian]

We are currently building with Clang 18 on Android. For this, you currently need to disable xnn_enable_avxvnniint8:

echo "build --define=xnn_enable_avxvnniint8=false" >> .bazelrc

This should allow you to create a custom LLM inference engine with Bazel 6.5.0.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No