ndk
ndk copied to clipboard
android
[FR] macOS & Apple Silicon Support
At WWDC 2020, Apple announced that macOS will be migrating to "Apple Silicon" over the next couple of years. While Rosetta 2 will exist as an immediate solution to allow the execution of existing x86_64 code on devices with Apple Silicon CPUs, the mid-to-long term plan should be to distribute all native binaries as Universal 2 binaries: executables which contain both x86_64 and Apple Silicon slices:
https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary
All native binaries in the Android SDK & NDK, such as clang and the linker, presumably need to be built as Universal 2 binaries.
Are there any plans to distribute the NDK as Universal 2 binaries?
Is your feature request related to a problem? Please describe.
Apple will be migrating macOS to a new hardware architecture. It would be nice if it were properly supported before x86 support is dead. (Admittedly not "soon"…)
Describe the solution you'd like
Ship NDK binaries as Universal 2 binaries, containing both x86_64 and Apple Silicon support.
Describe alternatives you've considered
Crying.
Components that need to be made fat
- [x] https://github.com/android/ndk/issues/1410
- [x] https://github.com/android/ndk/issues/1544
- [x] https://github.com/android/ndk/issues/1545
- [x] https://github.com/android/ndk/issues/1546
- [x] https://github.com/android/ndk/issues/1547
- [x] https://github.com/android/ndk/issues/1548
- [ ] https://github.com/android/ndk/issues/1549
- [x] https://github.com/android/ndk/issues/1577
- [x] https://github.com/android/ndk/issues/1584
Won't fix:
- GDB
- GAS
- renderscript
Thanks for filing. We've been planning for this since the rumors started being published, but hadn't filed a tracking bug.
We're working on getting some of the development devices. We also need to get buildbots that have a new enough xcode (which could possibly require a new version of macOS, I don't know). Some test infra work probably needs to be done, since I'm assuming that the new OS will be cross compiled for rather than natively compiled for.
After we have all the infrastructure we can finally start actually working on this. There may be some porting work, but for the most part we're shipping third-party software and will be dependent on those projects being ready.
In other words, there's a pretty long tail of work here, but we have started on it. There's still a non-zero chance that we'll need to rely on Rosetta at first. There is an outstanding question of whether or not Rosetta will support bare executables or just app bundles.
Crying.
Same.
The chances are pretty high that x86_64 NDK will run decently in Rosetta-2 (translation) mode even on the ARM hardware.
We can confirm that the x86_64 NDK tools that Xamarin.Android uses run on Rosetta 2 on the Apple DTK. I can't say how "well" they run, but they do run, and we're able to use Rosetta 2-translated SDK & NDK utilities to build & install a Xamarin.Android application.
Great, thanks for that info. Just to check, they run outside your app bundle as well?
Great, thanks for that info. Just to check, they run outside your app bundle as well?
No, we re-package and redistribute bits of the Android NDK toolchain such that they are inside our .pkg, e.g.:
/Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/ndk/aarch64-linux-android-ld
I don't have the DTK machine, so I can't verify that they run outside of our default installation location. I don't see why they wouldn't…
I would hope they do as well, but so far I haven't seen anything definitive about whether Rosetta will work for plain executables outside an app package. Our recent experience with notarization has me worried that it won't.
We should have our hands on the hardware soon enough and be able to find out though.
going back to the original question:
Are there any plans to distribute the NDK as Universal 2 binaries?
we're still thinking about whether we'll have universal binaries or two separate NDKs for darwin/x86-64 and darwin/arm64, so if you have a strong argument why you'd rather have universal binaries than a separate NDK, let us know while we/Studio are still trying to decide! we don't have any timeline to announce yet, but we have started work on building everything for darwin/arm64. (and in the meantime, everything works with Rosetta2.)
If you ask me, I'd prefer the smaller separate packages. These days, we mostly download NDK via SDK Manager, and it should know well which flavor of NDK is correct for the given machine.
BTW, same applies to Windows and Linux.
(and in the meantime, everything works with Rosetta2.)
How do you run ndk using rosetta2?
Nothing special I don't think. The whole point of Rosetta is to allow binaries to run as if nothing were different.
yeah, that's my understanding too (and how it was for "classic" Rosetta back in the PowerPC transition days). @kongy has actually tried it...
hi, I get this error while trying to build a project with ndk
Error while executing process /Users/osrl/Library/Android/sdk/ndk/21.3.6528147/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Android.mk NDK_APPLICATION_MK=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Application.mk APP_ABI=armeabi-v7a NDK_ALL_ABIS=armeabi-v7a NDK_DEBUG=1 APP_PLATFORM=android-18 NDK_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/obj NDK_LIBS_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n} ERROR: Unknown host CPU architecture: arm64
I also tried to build with arch -x86_64 but no luck. How does "everything work with Rosetta2"? Can you help @enh-google?
I solved it by editing ndk-build script:
#!/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
After that, ndk-build gave me "invalid option -- O" error
I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@
Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?
Presumably you're getting your system's make and not the NDK's when you make that change. If you're using an older make there are going to be some broken parts of ndk-build.
(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)
I solved it by editing ndk-build script:
#!/bin/sh DIR="$(cd "$(dirname "$0")" && pwd)" arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"After that, ndk-build gave me "invalid option -- O" error I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?
Thanks
(i don't have a macOS 11 machine to check, but 10.15 certainly only had make 3.81, which is too old to have -O, and i don't expect Apple to have updated to a GPL3 version of GNU Make...)
My macOS 11.2.1 machine with Xcode 12.2 still has GNU make 3.81.
Do you have Rosetta 2 installed? That will be needed to run the compiler, which is an Intel binary. Rosetta 2 isn't always installed automatically.
John
On Fri, Apr 9, 2021 at 7:14 AM H3c @.***> wrote:
how to build use cmake???
i get some error on M1
C/C++: /Users/xxx/Documents/THAILib/app/CMakeLists.txt debug|armeabi-v7a : CMake Error at /Users/xxx/Library/Android/sdk/cmake/3.18.1/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake:66 (message): The C compiler
"/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /Users/xxx/Documents/THAILib/app/.cxx/Debug/3t4w4qs6/armeabi-v7a/CMakeFiles/CMakeTmp
Run Build Command(s):/Users/xxx/Library/Android/sdk/cmake/3.18.1/bin/ninja cmTC_ef1bc && [1/2] Building C object CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o FAILED: CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o /Users/xxx/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --gcc-toolchain=/Users/h3c/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Users/xxx/Documents/android-ndk-r21e/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -fPIE -MD -MT CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_ef1bc.dir/testCCompiler.c.o -c testCCompiler.c ninja: build stopped: subcommand failed.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/android/ndk/issues/1299#issuecomment-816437689, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATCLA5X3RCLVXMCOIX25UKLTH2LNPANCNFSM4OJWV7QA .
I solved it by editing ndk-build script:
#!/bin/sh DIR="$(cd "$(dirname "$0")" && pwd)" arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"After that, ndk-build gave me "invalid option -- O" error I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?
same problem
Enumerated what I think are all the subtasks here and filed bugs for them (task list in the OP). LLVM is ~done (we just need to take the latest version), but iirc someone told me that a Rosetta process invoking a fat binary will result in another Rosetta process, so until CMake/ninja/make are updated that doesn't actually change anything (though it does for other build systems).
Enumerated what I think are all the subtasks here and filed bugs for them (task list in the OP). LLVM is ~done (we just need to take the latest version), but iirc someone told me that a Rosetta process invoking a fat binary will result in another Rosetta process, so until CMake/ninja/make are updated that doesn't actually change anything (though it does for other build systems).
Hi, aren't CMake, make, Ninja and yasm available for M1? At least the homebrew versions seem to be. https://formulae.brew.sh/formula/make https://formulae.brew.sh/formula/cmake https://formulae.brew.sh/formula/ninja https://formulae.brew.sh/formula/yasm
You're welcome to use those. We need to update the ones that we build ourselves.
You're welcome to use those.
How can we tell the NDK to use the "native" brew versions? Is this trivial or does it require digging into the internals. In general I would be fine to dig into the internals but it doesn't seem worth it since it's already "working", but slower
I solved it by editing ndk-build script:
#!/bin/sh DIR="$(cd "$(dirname "$0")" && pwd)" arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"After that, ndk-build gave me "invalid option -- O" error I edited build/ndk-build script and removed -O argument from the command:
$GNUMAKE -f $PROGDIR/core/build-local.mk "$@Can anyone tell me if I did something wrong? Why did I had to edit ndk-build?
I added the above configuration, but running ndk-build still returns the error: ERROR: Unknown host CPU architecture: arm64. Is there something I don't set?
I think https://github.com/android/ndk/issues/1410 is the thread you want. The fix is in the next release, and https://android-review.googlesource.com/c/platform/ndk/+/1855814 was the fix (in the other thread, but buried in the middle).
How long is this still going to take? Its been over a year now.. @google wtf ist wrong with you??
The fix is in the next release
@DanAlbert Do you know what the next is ? At least it's not 23.1.7779620
The fix is in the next release
@DanAlbert Do you know what he next is ? At least it's not 23.1.7779620
you can see on any given bug... if you look at https://github.com/android/ndk/issues/1410, say, you can see on the right-hand side that it's currently targeted at r23c and r24.
for dates for any given release, see https://github.com/android/ndk/wiki#release-schedule