[REQUEST]: Android x86_64 support

[aaron-stafford]

Is your feature request related to a problem?

I'm trying to use this plugin for an x86_64 Android device but the x86_64 binaries are not present in: https://github.com/Unity-Technologies/com.unity.webrtc/blob/develop/Runtime/Plugins/Android/libwebrtc.aar so it fails on device due to a missing binary.

I'm just wondering if you have an obvious path forward for this? I'm happy to create a pull request if that helps. Or happy to help engineer another solution.

Describe the solution you'd like

Ideally, the Android x86_64 binaries would also live in https://github.com/Unity-Technologies/com.unity.webrtc/blob/develop/Runtime/Plugins/Android/libwebrtc.aar

Ideally they would also be updated in step with the regular updates for all binaries as the code changes.

Describe alternatives you've considered

I built the binary myself and put it into libwebrtc.aar and everything was fine. I considered making a pull request for this, but I'm not sure if this is really the correct approach because I suspect there is a more automated approach to updating the binaries. And so that pull request would be overwritten by any subsequent rebuild of the binaries. Meaning that the scripts used to generate the binaries would also need to be updated.

Additional context

No response

[ghost]

FWIW: This is the link to my fork: https://github.com/aaron-stafford/com.unity.webrtc

I needed to make a number of changes to get the Android x86_64 build process to work. I might be missing something obvious, but that should be a consideration in this request too.

[karasusan]

@aaron-stafford We are welcome to your contribution! We need to fix this build script for building aar binary. https://github.com/Unity-Technologies/com.unity.webrtc/blob/develop/BuildScripts~/build_plugin_android.sh

Of course we also have to make libwebrtc.a for Android x86_64.

[ghost]

Thanks @karasusan , I can potentially help to provide a solution for that build script if no one else is working on it. Is there any particular environment that the script needs to run in? Or is it expected that it would work for all of the environments specified here: https://github.com/Unity-Technologies/com.unity.webrtc/blob/develop/Plugin~/README.md ? I was a little unclear if the environments specified in that last link are purely for building the webrtc native code, or if they are also intended to be used to build the plugin. Or are they purely for building the plugin? Right now my solution works on Ubuntu 20.04 for building both the webrtc native code and the plugin but I haven't tried on anything else.

[karasusan]

@aaron-stafford The shell script I posted are purely for building the plugin. libwebrtc build script is here. https://github.com/Unity-Technologies/com.unity.webrtc/blob/develop/BuildScripts~/build_libwebrtc_android.sh

Unfortunatelly, we cannot provide the build environment publicly because the build machines are internal resources. But we can say that we use Ubuntu20.04 to build the native plugin for Android ARM64.

[karasusan]

memo: WRS-16

[PlinChen]

You can get android native libraries here: M92_webrtc-android_x64_arm64_armv7a.zip Then， you can build your own aar with build_plugin_android.sh

[ghost]

@karasusan , in the contributing page (https://github.com/Unity-Technologies/com.unity.webrtc/blob/main/CONTRIBUTING.md), its states:

By making a pull request, you confirm you agree to the terms and conditions of the Apache 2.0 Contributor License Agreement (whether Individual or Corporate) and have signed and submitted such Contributor License Agreement to the Apache 2.0 Software Foundation

Is that right? Do I need to submit a contributor license agreement to Apache 2.0 Software Foundation? Seems odd. Just want to clarify.

[karasusan]

@aaron-stafford When you make the PR on the repository, You can see the page to sign the agreement on GitHub. You only need to push the button on the aggeement page.

[karasusan]

@aaron-stafford Any questions?

[ghost]

@karasusan , thanks. No more question on the legal side of things for now. This seems fine.

@PlinChen , it is my understanding that build_plugin_android.sh needs modification to build the x86_64 version of the plugin. Are you suggesting that you were able to use it unchanged to generate the plugin? I don't see how that is possible, I'd love to understand how you managed it.

[lumin-leaper]

I think I might be able to provide a solution for this. From what I can tell, build_libwebrtc_android.sh just needs to be modified to have its target_cpu changed to include both "arm64" and "x64". This would put the binaries for both architectures into the same webrtc-android.zip file.

If this file were to be generated and upload by what ever build system you have, it could then be referenced by the build_plugin_android.sh script.

build_plugin_android.sh would need a bit more work as it appears to only build for a single architecture presently, so it would need to be modified to build for both arm64-v8a and x86_64. This would generate result in a new libwebrtc.aar file that would contain all the binary code for both architectures.

This all seems fairly trivial, if I provide the changes for this, would a solution like that be acceptable? The main issue that I see is that the URL that the build_plugin_android.sh references could not be known to me at the time I generate a pull request. I guess one solution might be to provide a PR just for the build_libwertc_android.sh initially, and once that is working and a URL to the binary is generated, I could provide a PR to the build_plugin_android.sh?

Please let me know what you think. I'd love to help out on this if I can.

[karasusan]

@lumin-leaper

As you mentioned, to solve this issue, we need to modify build_libwebrtc_android.sh to create a static library for x86_64. this script is ran on our in-house CI system to build the library. The library will be uploaded to GitHub Release.

If we want to create a static library, it is sufficient to modify build_libwebrtc_android.sh, but to build the Unity native plugin, we need to tell the linker the location of the static library. I think this will require an update to FindWebRTC.cmake. Additionally, you will need to specify the Android architecture in build_plugin_android.sh.

[lumin-leaper]

@karasusan , thanks for the tip regarding FindWebRTC.cmake, a small change to this file indeed makes it easier for build_plugin_android.sh to find the library:

diff --git a/Plugin~/cmake/FindWebRTC.cmake b/Plugin~/cmake/FindWebRTC.cmake
index 55fe5b3..b812c2e 100644
--- a/Plugin~/cmake/FindWebRTC.cmake
+++ b/Plugin~/cmake/FindWebRTC.cmake
@@ -26,6 +26,9 @@ if(Windows OR Linux)
   set(WEBRTC_LIBRARY_DIR ${WEBRTC_LIBRARY_DIR}/${SYSTEM_PROCESSOR})
 endif()
 if(Android)
+  if(CMAKE_ANDROID_ARCH_ABI STREQUAL "x86_64")
+    set(CMAKE_ANDROID_ARCH "x64")
+  endif()
   set(WEBRTC_LIBRARY_DIR ${WEBRTC_LIBRARY_DIR}/${CMAKE_ANDROID_ARCH})
 endif()
 

This makes the change to build_plugin_android.sh a bit simpler:

diff --git a/BuildScripts~/build_plugin_android.sh b/BuildScripts~/build_plugin_android.sh
index fb0cff8..4f23d85 100755
--- a/BuildScripts~/build_plugin_android.sh
+++ b/BuildScripts~/build_plugin_android.sh
@@ -3,7 +3,6 @@
 export LIBWEBRTC_DOWNLOAD_URL=https://github.com/Unity-Technologies/com.unity.webrtc/releases/download/M107/webrtc-android.zip
 export SOLUTION_DIR=$(pwd)/Plugin~
 export PLUGIN_DIR=$(pwd)/Runtime/Plugins/Android
-export ARCH_ABI=arm64-v8a
 
 # Download LibWebRTC 
 curl -L $LIBWEBRTC_DOWNLOAD_URL > webrtc.zip
@@ -11,25 +10,29 @@ unzip -d $SOLUTION_DIR/webrtc webrtc.zip
 
 # Build UnityRenderStreaming Plugin 
 cd "$SOLUTION_DIR"
-cmake . \
-  -B build \
-  -D CMAKE_SYSTEM_NAME=Android \
-  -D CMAKE_ANDROID_API_MIN=24 \
-  -D CMAKE_ANDROID_API=24 \
-  -D CMAKE_ANDROID_ARCH_ABI=$ARCH_ABI \
-  -D CMAKE_ANDROID_NDK=$ANDROID_NDK \
-  -D CMAKE_BUILD_TYPE=Release \
-  -D CMAKE_ANDROID_STL_TYPE=c++_static
+for ARCH_ABI in "arm64-v8a" "x86_64"
+do
+  cmake . \
+    -B build \
+    -D CMAKE_SYSTEM_NAME=Android \
+    -D CMAKE_ANDROID_API_MIN=24 \
+    -D CMAKE_ANDROID_API=24 \
+    -D CMAKE_ANDROID_ARCH_ABI=$ARCH_ABI \
+    -D CMAKE_ANDROID_NDK=$ANDROID_NDK \
+    -D CMAKE_BUILD_TYPE=Release \
+    -D CMAKE_ANDROID_STL_TYPE=c++_static
 
-cmake \
-  --build build \
-  --target WebRTCPlugin
+  cmake \
+    --build build \
+    --target WebRTCPlugin
 
-# libwebrtc.so move into libwebrtc.aar
-cp -f $SOLUTION_DIR/webrtc/lib/libwebrtc.aar $PLUGIN_DIR
-pushd $PLUGIN_DIR
-mkdir -p jni/$ARCH_ABI
-mv libwebrtc.so jni/$ARCH_ABI
-zip -g libwebrtc.aar jni/$ARCH_ABI/libwebrtc.so
-rm -r jni
-popd
\ No newline at end of file
+  # libwebrtc.so move into libwebrtc.aar
+  cp -f $SOLUTION_DIR/webrtc/lib/libwebrtc.aar $PLUGIN_DIR
+  pushd $PLUGIN_DIR
+  mkdir -p jni/$ARCH_ABI
+  mv libwebrtc.so jni/$ARCH_ABI
+  zip -g libwebrtc.aar jni/$ARCH_ABI/libwebrtc.so
+  rm -r jni
+  popd
+  rm -rf build
+done
\ No newline at end of file

The diff makes it look more complicated than it is, the majority of the code is just put into a for loop to iterate over both architectures. Notably, I also remove the build folder between builds. This may be unnecessary, but I did it out of an abundance of caution. Also, hopefully obviously, this wouldn't be the final patch because LIBWEBRTC_DOWNLOAD_URL still references the original url. As mentioned previously, a new version of libwebrtc would need to be generated and uploaded before the correct url could be specified here.

To build the native library, the changes to build_libwebrtc_android.sh are trivial as thought:

diff --git a/BuildScripts~/build_libwebrtc_android.sh b/BuildScripts~/build_libwebrtc_android.sh
index 05b76f2..d571042 100755
--- a/BuildScripts~/build_libwebrtc_android.sh
+++ b/BuildScripts~/build_libwebrtc_android.sh
@@ -41,7 +41,7 @@ popd
 
 mkdir -p "$ARTIFACTS_DIR/lib"
 
-for target_cpu in "arm64"
+for target_cpu in "arm64" "x64"
 do
   mkdir -p "$ARTIFACTS_DIR/lib/${target_cpu}"
 
@@ -83,7 +83,7 @@ do
   "$PYTHON3_BIN" tools_webrtc/android/build_aar.py \
     --build-dir $OUTPUT_DIR \
     --output $OUTPUT_DIR/libwebrtc.aar \
-    --arch arm64-v8a \
+    --arch arm64-v8a x86_64 \
     --extra-gn-args "is_debug=${is_debug} \
       is_java_debug=${is_debug} \
       rtc_use_h264=false \

Should I put these changes to build_libwebrtc_android.sh into a Pull Request so that the library can be generated? Once that is done I could provide the final changes to build_plugin_android.sh.

[karasusan]

@lumin-leaper Sure, please make the pull request for the patch of build_libwebrtc_android.sh. If so, we will update the library on the GitHub.

[JacopoMarullo]

Hello, I am also interested in using Unity WebRTC for Android x86_64 (specifcally for Magic Leap2). Is there any activity in progress on this side? @lumin-leaper did you finally submit your PR?

Thanks a lot Jacopo

[lumin-leaper]

@JacopoMarullo , I'm just waiting on the green light from my company's legal team. The company as a whole is going through a process to re-evaluate all open source project contributions. It's taken a bit longer than I thought it might, but I think we are close to having it all sorted out. Thanks for poking the thread. It gives me the opportunity to escalate it again.

[JacopoMarullo]

Thanks a lot for the clarification! Please, keep me update if you there are some news about it.

Thanks again Jacopo

[karasusan]

@lumin-leaper @JacopoMarullo Hi, if we test the architecture Android X86, which device do you prefer to test for? I haven't tested that arch in our environment.

[lumin-leaper]

@karasusan , I'm targeting the Magic Leap 2 device. From @JacopoMarullo comments, it looks like we are targeting the same device.

[JacopoMarullo]

Hi guys, yes I'm targeting the Magic Leap 2 device

[crucial-sailor]

@lumin-leaper do you have any updates on this? My team would also be interested in using the library on an x64 Android device.

[lumin-leaper]

I have branch ready to create the pull request from: https://github.com/lumin-leaper/com.unity.webrtc/tree/issue-802-part-1

However, it looks like the tip of mainline is not working the same for me compared to when I first built this code several months ago. I now need the following additional change for build_libwebrtc_android.sh to run correctly:

diff --git a/BuildScripts~/build_libwebrtc_android.sh b/BuildScripts~/build_libwebrtc_android.sh
index 05b76f2..b74d6c0 100755
--- a/BuildScripts~/build_libwebrtc_android.sh
+++ b/BuildScripts~/build_libwebrtc_android.sh
@@ -2,7 +2,10 @@
 
 if [ ! -e "$(pwd)/depot_tools" ]
 then
-  git clone --depth 1 https://chromium.googlesource.com/chromium/tools/depot_tools.git
+  git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
+  pushd depot_tools
+  git reset --hard 96b57b9b59091007162adfa4035a6383e9757815
+  popd
 fi
 
 export COMMAND_DIR=$(cd $(dirname $0); pwd)
@@ -11,6 +14,7 @@ export WEBRTC_VERSION=5304
 export OUTPUT_DIR="$(pwd)/out"
 export ARTIFACTS_DIR="$(pwd)/artifacts"
 export PYTHON3_BIN="$(pwd)/depot_tools/python-bin/python3"
+export DEPOT_TOOLS_UPDATE=0
 
 if [ ! -e "$(pwd)/src" ]
 then

To me it appears as though recent changes in depot_tools is not compatible with the tip of this repo. The change above simply pegs depot_tools to a known working version.

So I'm a bit unsure about how to proceed at the moment. @karasusan , are you aware of this issue? Should I raise a bug for it? Or is it not actually a problem in your build environment? Should I submit my PR any way?

[kannan-xiao4]

@lumin-leaper The dependency of depot tools is now described in the webrtc library, but at present it is not possible to follow it. So, libwebrtc upgrade is required and we proceed that.

[lumin-leaper]

@kannan-xiao4, just to clarify, you are saying that work will be done to get libwebrtc building with the head of depot_tools prior to considering a pull request from me?

[kannan-xiao4]

@lumin-leaper I think the changes you're showing will work correctly on the current branch. However, I think that if we upgrade from M107 to M112 libwebrtc, it will be unnecessary code. We have confirmed that upgrading to M112 will fix the depot_tools issue.

[lumin-leaper]

Thanks @kannan-xiao4 I see that libwebrtc builds cleanly now with modification. I've submitted the first pull request as discussed in this thread: https://github.com/Unity-Technologies/com.unity.webrtc/pull/932 I look forward to the feedback.

[crucial-sailor]

@lumin-leaper Is there anything else you did to make the generated .aar file work on the MagicLeap 2?

After copying the required project folders from the repo (Runtime (which contains the newly generated .aar), Editor and Tests folder, as well as the package.json), importing the folder in Unity and starting the project on the MagicLeap, the app stays blank / seems to be broken. The same behaviour could not be observed when the "original" webrtc package is used instead (even though that one does not actually run on the MagicLeap 2 of course).

For us, the build process went fine, and the .aar file also seems to contain all necessary files when compared to the one in the webrtc_android.zip file, which can be downloaded in Releases. Could it be that some import settings are wrong, or the build is broken, even though it does not report to be so?

[lumin-leaper]

@oculavis-cvar , I'd need to know more. Where are you getting your webrtc_android.zip and .aar files from? Are these all generated yourself from the information above? Or are you attempting to use files generated from the pull request?

With respect to the pull request I made yesterday, it is only the first part. It allows libwebrtc to be built. But there is a second pull request that will be needed to actually generate the plugin (.aar file) correctly.

If you generated it yourself from the above information, you need to be a bit careful with the generation of the plugin (which is why this is broken up into two pull requests). The code to generate the plugin that I posted earlier in this thread, if used literally, will result in a slightly broken .aar file. The build_plugin_android.sh file will need to look a bit more like this: https://github.com/lumin-leaper/com.unity.webrtc/blob/issue-802-part-2-build-test/BuildScripts~/build_plugin_android.sh (that is, note the commenting out of the curl command). Also in my original post above, technically there is an error (note the position of this line of code: cp -f $SOLUTION_DIR/webrtc/lib/libwebrtc.aar $PLUGIN_DIR). It needs to be moved out of the loop otherwise it'll overwrite previous loops. The correct implementation is also in that same link. That link is just for testing locally though. Due to commenting out the curl command it can't be part of the pull request.

If you are still seeing issues, please explain further how you are generating the files for the plugin.

[crucial-sailor]

@lumin-leaper Currently, we only build the x64 version of the plugin. We started with your proposed changes to FindWebRTC.cmake and build_libwebrtc_android.sh, but only building for x64 Android. The build worked fine, we did not see any error messages, and the resulting webrtc-android.zip looked fine too. We extracted its contents manually to the folder required by build_plugin_android.sh, then started the build script, again just for x86_64 as a target.

The resulting output is written to Runtime/Plugins/Android/libwebrtc.aar. Given the structure of the Unity plugin, we then copied all folders relevant to import into Unity in a "com.unity.webrtc" folder, including the full Runtime, Documentation~, Editor, Samples~ and Tests folder, and the package.json file, all from the repository. Then, we imported this folder in Unity. Finally, we built for the MagicLeap. There were no build errors, and also no code errors in the Editor.

Still, the MagicLeap app is broken as soon as the plugin is loaded. Without it, it works fine. With it, the app starts and stays blank. No loading screen. No Unity logo. Nothing. It does not even crash, or produce any log in the console or anything useful in logcat.

That's why we decided to post here, because we are a bit at a loss. Below I'll post the only file changes we've made in the repository (they're not very clean, but should work for testing locally). The only potential problem I could come up with is that we can't just take the files from the repository to put together the plugin to create for Unity.

diff --git a/BuildScripts~/build_libwebrtc_android.sh b/BuildScripts~/build_libwebrtc_android.sh
index 36d6e43..b00bc61 100755
--- a/BuildScripts~/build_libwebrtc_android.sh
+++ b/BuildScripts~/build_libwebrtc_android.sh
@@ -50,7 +50,7 @@ popd
 mkdir -p "$ARTIFACTS_DIR/lib"
 
 
-for target_cpu in "arm64"
+for target_cpu in "x64"
 do
   mkdir -p "$ARTIFACTS_DIR/lib/${target_cpu}"
 
@@ -94,7 +94,7 @@ do
   "$PYTHON3_BIN" tools_webrtc/android/build_aar.py \
     --build-dir $OUTPUT_DIR \
     --output $OUTPUT_DIR/libwebrtc.aar \
-    --arch arm64-v8a \
+    --arch x86_64 \
     --extra-gn-args "is_debug=${is_debug} \
       is_java_debug=${is_debug} \
       rtc_use_h264=false \
diff --git a/BuildScripts~/build_plugin_android.sh b/BuildScripts~/build_plugin_android.sh
index 04873c1..1c42fec 100755
--- a/BuildScripts~/build_plugin_android.sh
+++ b/BuildScripts~/build_plugin_android.sh
@@ -3,14 +3,18 @@
 export LIBWEBRTC_DOWNLOAD_URL=https://github.com/Unity-Technologies/com.unity.webrtc/releases/download/M112/webrtc-android.zip
 export SOLUTION_DIR=$(pwd)/Plugin~
 export PLUGIN_DIR=$(pwd)/Runtime/Plugins/Android
-export ARCH_ABI=arm64-v8a
+export ARCH_ABI=x86_64
 
 # Download LibWebRTC 
-curl -L $LIBWEBRTC_DOWNLOAD_URL > webrtc.zip
-unzip -d $SOLUTION_DIR/webrtc webrtc.zip 
+# TODO INSTEAD COPY OVER THE RESULTS OF build_libwebrtc_android.sh
+# Make sure the keep the same folder structure!
+# curl -L $LIBWEBRTC_DOWNLOAD_URL > webrtc.zip
+# unzip -d $SOLUTION_DIR/webrtc webrtc.zip 
 
 # Build UnityRenderStreaming Plugin 
 cd "$SOLUTION_DIR"
+# for ARCH_ABI in "arm64-v8a" "x86_64"
+# do
 cmake . \
   -B build \
   -D CMAKE_SYSTEM_NAME=Android \
@@ -32,4 +36,6 @@ mkdir -p jni/$ARCH_ABI
 mv libwebrtc.so jni/$ARCH_ABI
 zip -g libwebrtc.aar jni/$ARCH_ABI/libwebrtc.so
 rm -r jni
-popd
\ No newline at end of file
+popd
+# rm -rf build
+# done
\ No newline at end of file
diff --git a/Plugin~/cmake/FindWebRTC.cmake b/Plugin~/cmake/FindWebRTC.cmake
index 55fe5b3..9c9a7f6 100644
--- a/Plugin~/cmake/FindWebRTC.cmake
+++ b/Plugin~/cmake/FindWebRTC.cmake
@@ -26,6 +26,7 @@ if(Windows OR Linux)
   set(WEBRTC_LIBRARY_DIR ${WEBRTC_LIBRARY_DIR}/${SYSTEM_PROCESSOR})
 endif()
 if(Android)
+  set(CMAKE_ANDROID_ARCH "x64")
   set(WEBRTC_LIBRARY_DIR ${WEBRTC_LIBRARY_DIR}/${CMAKE_ANDROID_ARCH})
 endif()

[lumin-leaper]

@oculavis-cvar , its a bit hard to say for sure with what you posted. Some of the changes are a little bit different to what I posted, but mostly it looks correct. This does not look correct though. +# for ARCH_ABI in "arm64-v8a" "x86_64" This loop will also try to include the arm binaries. Given your comment about only building for x64, i'm not sure what would happen if they didn't exist. I presume the script would error out. But perhaps not.

This is also a bit ambiguous:

+# TODO INSTEAD COPY OVER THE RESULTS OF build_libwebrtc_android.sh
+# Make sure the keep the same folder structure!
+# curl -L $LIBWEBRTC_DOWNLOAD_URL > webrtc.zip
+# unzip -d $SOLUTION_DIR/webrtc webrtc.zip 

You might be doing the right thing here, but it's hard to tell for sure. If you unzip the final version of Runtime/Plugins/Android/libwebrtc.aar and print the folder/file structure it might help to confirm it is okay. I'd also suggest running objdump on the binaries that you built to confirm that they were actually built for x86_64 (I managed once to generate x86_64 names files that were actually arm). Although if this were the case, you should see a message in logcat that the webrtc library failed to load (it doesn't crash the app though).

we then copied all folders relevant to import into Unity in a "com.unity.webrtc" folder I'm a bit unclear if you mean you are using the folders as the basis for a new unity project or if you are still using the artifacts as a plugin. I've only ever used these artifacts as a plugin. So i'm not sure what would happen if you tried to turn it directly into a project.

The only other thing that comes to mind (if all of the above is ok) is that you have correctly added all of the scene to your build target. I suspect if you didn't add any scenes, you might get behavior like what you describe, but again. I'm not sure. It may or may not be obvious, but your build settings need to target Android, you need to build for il2cpp and you need to target x86_64.

Hopefully that helps. If not, you could try regenerating the plugin using the script I have in this branch: https://github.com/lumin-leaper/com.unity.webrtc/tree/issue-802-part-2-build-test It is designed to build the plugin from a custom artifact rather than from the downloaded url. You just need to copy the webrtc-android.zip file (generated from the pervious libwebrtc script) and it'll do the rest. This would eliminate any difference in our process.