[BUG] Not providing a zip download for Mac anymore breaks lots of CI systems

[a4z]

Description

There has always been a zip file download of the NDK for Mac, but now only a dmg file download exists.

Please see the current r23 download: https://developer.android.com/ndk/downloads#lts-downloads Up to r22b, there was always a zip file to download.

Not providing the zipfile anymore breaks all CI builds that make use of the zip package, and complicates automated extracting of the NDK unnecessarily.

hdiutil attach is a multi step operation, that requires mounting, getting files, and un-mounting of the dmg package. That is a real error prone way and a complicated worsening for Mac users.

It would be nice if the zip file could be added again to keep the existing simplicity when getting the NDK in an automated way on Mac

Environment Details

• NDK Version: r23
• Build system: not relevant
• Host OS: Mac
• ABI: not relevant
• NDK API level: not relevant
• Device API level: not relevant

[DanAlbert]

The zip download link was removed from the page because users were filing bugs because they downloaded the zip when they needed the app bundle, but didn't understand the difference. The zip doesn't work for most scenarios because macOS requires a notarized package (it's not possible to notarize the zip NDK), whereas the app bundle works for all scenarios (aside from the SDK manager) so the zip download was assumed to be more or less useless for anyone visiting that page.

The zip download still exists: https://dl.google.com/android/repository/android-ndk-r23-darwin.zip, it's just not shown on the page to avoid confusing users. Given that our URLs are predictable, is that sufficient, or do we need to put the explicit download link back?

A few more questions:

1. How are you using the unnotarized package? Are you disabling/bypassing gatekeeper in CI?
2. Why are you using macOS for CI? I'm not saying you're doing it wrong, I'm just curious what I'm missing that makes that a better option over Linux.

[a4z]

Thanks you so much for the link!

I do not know or understand what you mean with Gatekeeper problems and notarized package.

I, and some other user around the world, used the NDK to compile some C++ dependencies. That just works. We use the conan package as a build requirement for conan, and build C++ libraries for Android, which our Apps consume. There has never been a problem using that package. https://conan.io/center/android-ndk

The last place where I personally used macOS in the CI uses it because we also build iOS packages of the same C++ core components, and to not have to maintain 2 different systems, all development and build happens on MacOS , using the Mac NDK from the zip, without a problem. That works quite nice. I do not say that this is a better option over Linux, but on a place where everyone knows MacOS and no one knows Linux, it is.

Thanks again for the link, this is important to all people who have a similar reasons to use the NDK from the zip like I. Please keep the zip available!

[DanAlbert]

I do not know or understand what you mean with Gatekeeper problems and notarized package.

I, and some other user around the world, used the NDK to compile some C++ dependencies. That just works. We use the conan package as a build requirement for conan, and build C++ libraries for Android, which our Apps consume. There has never been a problem using that package. https://conan.io/center/android-ndk

Ah, right, because because conan is the thing performing the download. This works around the notarization requirement the same way the SDK does, but not telling gatekeeper that it needs to care.

Gatekeeper only checks for notarization for files that were downloaded from the internet, and unless the downloading program intentionally flags files they won't be checked. Chrome, Safari, etc all mark files for gatekeeper, but (at least last I checked) things like curl and wget do not (which means that probably any CI setup script is going to be unaffected, at least until curl/wget change or apple makes gatekeeper more strict).

The last place where I personally used macOS in the CI uses it because we also build iOS packages of the same C++ core components, and to not have to maintain 2 different systems, all development and build happens on MacOS , using the Mac NDK from the zip, without a problem. That works quite nice. I do not say that this is a better option over Linux, but on a place where everyone knows MacOS and no one knows Linux, it is.

Understood, thanks for explaining. Will try to keep that in mind.

Please keep the zip available!

We can't remove the zip, FWIW. The SDK depends on it because it, like conan, doesn't speak DMG.

Leaving this open to try putting the zip links back in a way that's less confusing to folks that need the app bundle.

[dmn-star]

Why are you using macOS for CI? I'm not saying you're doing it wrong, I'm just curious what I'm missing that makes that a better option over Linux.

used macOS in the CI uses it because we also build iOS packages of the same C++ core components, and to not have to maintain 2 different systems, all development and build happens on MacOS , using the Mac NDK from the zip, without a problem. That works quite nice. I do not say that this is a better option over Linux, but on a place where everyone knows MacOS and no one knows Linux, it is.

Same here. And all our devs use macs for mobile iOS/Android development.

[dmn-star]

@DanAlbert

I have tested the update to NDK r23 locally on my workstation and it does not work yet.

r22b works as expected. Any suggestions how I can fix this?

[a4z]

confirmed, first I thought, wrong permission

[a4z@mach2:~/.../llvm/prebuilt/darwin-x86_64/bin] (20:42:07)
-> pwd
/Users/a4z/.conan/data/android-ndk/r23/_/_/package/46f53f156846659bf39ad6675fa0ee8156e859fe/toolchains/llvm/prebuilt/darwin-x86_64/bin

[a4z@mach2:~/.../llvm/prebuilt/darwin-x86_64/bin] (20:42:17)
-> ls -l clang*
-rw-r--r--  1 a4z  staff         8 Aug 23 20:40 clang
-rw-r--r--  1 a4z  staff         5 Aug 23 20:40 clang++
-rwxr-xr-x  1 a4z  staff  75426096 Aug 23 20:40 clang-12
-rwxr-xr-x  1 a4z  staff  53376480 Aug 23 20:40 clang-check
-rwxr-xr-x  1 a4z  staff   1643296 Aug 23 20:40 clang-format
-rwxr-xr-x  1 a4z  staff   2950840 Aug 23 20:40 clang-tidy
-rwxr-xr-x  1 a4z  staff  35362896 Aug 23 20:40 clang-tidy.real
-rwxr-xr-x  1 a4z  staff  28783232 Aug 23 20:40 clangd

surprising size

[a4z@mach2:~/.../llvm/prebuilt/darwin-x86_64/bin] (20:42:20)
-> file clang
clang: ASCII text, with no line terminators

turns out, those are text files .... (clang and clang++)

[a4z@mach2:~/.../llvm/prebuilt/darwin-x86_64/bin] (20:43:23)
-> cat clang 
clang-12%                                                                                                                                                                    
[a4z@mach2:~/.../llvm/prebuilt/darwin-x86_64/bin] (20:43:38)
-> cat clang++ 
clang%  

I wonder if that is something that should be reported to google

[dmn-star]

Good catch. :-)

/Users/dmn/.conan/data/android-ndk/r23///package/46f53f156846659bf39ad6675fa0ee8156e859fe/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang --version

Permission denied.

Maybe a broken symlink? clang++ -> clang12

[enh-google]

works for me from the zip download link above, on macOS 11.5.1...

if i unzip and then immediately

~/Downloads/android-ndk-r23$ ./toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang --version

then at this point the "untrusted code" dialog box appears, and i hit cancel rather than move to trash...

./toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang: line 7:  4819 Killed: 9               `dirname $0`/clang --target=aarch64-linux-android21 "$@"

but as long as i remove the quarantine flag, it works fine:

~/Downloads/android-ndk-r23$ xattr -r -d com.apple.quarantine ~/Downloads/android-ndk-r23
~/Downloads/android-ndk-r23$ ./toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang --version
Android (7284624, based on r416183b) clang version 12.0.5 (https://android.googlesource.com/toolchain/llvm-project c935d99d7cf2016289302412d708641d52d2f7ee)
Target: aarch64-unknown-linux-android21
Thread model: posix
InstalledDir: /Users/enh/Downloads/android-ndk-r23/./toolchains/llvm/prebuilt/darwin-x86_64/bin
~/Downloads/android-ndk-r23$ 

the original zip file looks correct too:

~/Downloads$ zipinfo android-ndk-r23-darwin.zip | grep aarch64-linux-android21-clang
-rwxr-xr-x  3.0 unx      178 tx defX 21-Jul-30 17:08 android-ndk-r23/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang++
-rwxr-xr-x  3.0 unx      174 tx defX 21-Jul-30 17:08 android-ndk-r23/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang
~/Downloads$ 

the links are links for me too (to a binary that exists and has the right permissions):

lrwxr-xr-x  1 enh  primarygroup         8 Aug 23 12:47 clang -> clang-12
lrwxr-xr-x  1 enh  primarygroup         5 Aug 23 12:47 clang++ -> clang
-rwxr-xr-x  1 enh  primarygroup  75426096 Jul 30 17:06 clang-12

how exactly did you extract the zip file? seems like something went wrong with that?

[DanAlbert]

(In any case, if you do find a problem, file a new bug. This bug is about the missing URLs, and GitHub doesn't let me manipulate half of a bug so mixing them makes them hard to keep track of)

[dmn-star]

@enh-google The Conan extract the zip file for me, I think with https://docs.python.org/3/library/zipfile.html I'll open an issue in conan repo. Thanks.

@DanAlbert sorry, I will do next time in any case.

[DanAlbert]

http://cl/666538358 updates the text on the download page to explain that the zip artifacts haven't gone away. Should be live tomorrow.