[Android Editor] Add support for exporting platform binaries

[m4gr3d]

trafficstars

This PR addresses one of the known limitations of the Godot Android Editor by adding support for generating and exporting Android APK and other platform binaries straight from the Android Editor!

• 1st commit: Add support for the Android-specific assets path prefix

  • This is used to store and access the default debug.keystore used for signing the Android debug builds
  • See here for a full description of the change.

• 2nd commit: Enable support for platform binaries generation:

  • Enables Android apk signing (and verification) using the apksig library from https://android.googlesource.com/platform/tools/apksig/+/ac5cbb07d87cc342fcf07715857a812305d69888
  • Updates export logic to enable platform exports from the Android editor for the following platforms:
    • Android APK exports
      • Note: Only Android legacy builds are supported. Android gradle builds (and AAB exports) are not supported at this point in time.
    • Linux exports
    • Web exports
    • Unsigned Windows exports
    • MacOS exports without notarization and with ad-hoc signature
    • iOS project only exports (limitation for all non-macOS platforms)

Closes https://github.com/godotengine/internal-team-priorities/issues/88 Supersedes https://github.com/godotengine/godot/pull/89412 Addresses feedback from https://github.com/godotengine/godot/issues/73130

https://github.com/godotengine/godot/assets/914968/61a2e702-2458-44ab-ab51-141019d47985

[m4gr3d]

@adamscott @akien-mga Have we considered the option of breaking down the export template per platform?

To test the new export capability, I had to download the full export template (~1GB in size) on the Android tablet running the Godot editor, and I ended up using less than 10% of it given only the Android templates are used to generate the Android binaries (and no other platform binaries is supported on the Android editor).

We could instead break down the export template per platform, and provide users with a set of checkbox to allow them to select the templates for the platforms they are targeting.

[Calinou]

@adamscott @akien-mga Have we considered the option of breaking down the export template per platform?

Yes, see https://github.com/godotengine/godot-proposals/issues/647. It only needs minimal editor changes, as manual TPZ installation already works if a TPZ only contains certain platforms (or even some binaries of a given platform). The main part is doing the buildsystem changes so that there's an official TPZ per platform.

(and no other platform binaries is supported on the Android editor).

Supporting Windows/Linux/web exports should be trivial as it's just copying a bunch of files (and creating a ZIP archive in the case of web exports). It won't be possible to run rcedit to change the Windows application icon, but it's not essential.

[m4gr3d]

It won't be possible to run rcedit to change the Windows application icon, but it's not essential.

@Calinou For Windows we also lose the ability to sign the generated binary as described in https://docs.godotengine.org/en/stable/tutorials/export/exporting_for_windows.html#code-signing.

But yes, it looks like we should also be able to support linux and web. I'll give it a try.

[bruvzg]

Supporting export for macOS (without notarization and with ad-hoc signature) and iOS project only (limitation for all non-macOS platforms) should also be also trivial (require only zip and file manipulations).

[m4gr3d]

Supporting export for macOS (without notarization and with ad-hoc signature) and iOS project only (limitation for all non-macOS platforms) should also be also trivial (require only zip and file manipulations).

@bruvzg MacOS and iOS work as expected (with the listed limitations). I've updated the list of supported platforms accordingly.

[m4gr3d]

@akien-mga commit 39f99a333daee47841d9e2adae017937af71aaf0 is a follow-up to https://github.com/godotengine/godot/pull/93551 and updates the visibility of the custom_template options for all platforms for Godot 4.4

[dsnopek]

I tested this PR on a Huion Kamvas Slate 10, but unfortunately, I'm getting "Code Signing: Unable to sign apk." when attempting to export to Android (using a custom built Android APK template that I uploaded to the device).

I put the template APK in the "Documents" directory, similar to what is shown in the video.

The debug.keystore appears to exist at assets://keystores/debug.keystore, I assume created automatically.

The file dialog acts a little strange when I put assets:// into the path at the top - it automatically changes it to /assets:// - it shows the right files, but I can't seem to navigate around within the folder.

Am I missing any steps?

[m4gr3d]

I tested this PR on a Huion Kamvas Slate 10, but unfortunately, I'm getting "Code Signing: Unable to sign apk." when attempting to export to Android (using a custom built Android APK template that I uploaded to the device).

I put the template APK in the "Documents" directory, similar to what is shown in the video.

The debug.keystore appears to exist at assets://keystores/debug.keystore, I assume created automatically.

The file dialog acts a little strange when I put assets:// into the path at the top - it automatically changes it to /assets:// - it shows the right files, but I can't seem to navigate around within the folder.

Am I missing any steps?

@dsnopek There should be additional logs in logcat with a tag of ApkSignerUtil that specify why the signing step failed. Can you post them as well.

The debug.keystore appears to exist at assets://keystores/debug.keystore, I assume created automatically.

The debug.keystore is automatically included in the editor's apk, so that by default users can export debug apks without having to generate a keystore.

The file dialog acts a little strange when I put assets:// into the path at the top - it automatically changes it to /assets:// - it shows the right files, but I can't seem to navigate around within the folder.

assets:// should not be accessible via the file dialog; why do you need to navigate around that folder?

[dsnopek]

@dsnopek There should be additional logs in logcat with a tag of ApkSignerUtil that specify why the signing step failed. Can you post them as well.

Here's the log messages:

10746:06-25 13:10:09.645 12676 12704 V ApkSignerUtil: Loading keystore assets://keystores/debug.keystore with type PKCS12
10747:06-25 13:10:09.646 12676 12704 E ApkSignerUtil: Unable to load the keystore from assets://keystores/debug.keystore
10748:06-25 13:10:09.646 12676 12704 E ApkSignerUtil: java.io.IOException: error constructing MAC: java.security.InvalidKeyException: No installed provider supports this key: com.android.org.bouncycastle.jcajce.PKCS12Key
10749:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at com.android.org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(PKCS12KeyStoreSpi.java:852)
10750:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at java.security.KeyStore.load(KeyStore.java:1484)
10751:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.utils.ApkSignerUtil.signApk(ApkSignerUtil.kt:121)
10752:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.utils.ApkSignerUtil.signApk$default(ApkSignerUtil.kt:84)
10753:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.Godot.nativeSignApk(Godot.kt:1051)
10754:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.GodotLib.step(Native Method)
10755:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.gl.GodotRenderer.onDrawFrame(GodotRenderer.java:57)
10756:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.gl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1578)
10757:06-25 13:10:09.646 12676 12704 E ApkSignerUtil:   at org.godotengine.godot.gl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1280)

assets:// should not be accessible via the file dialog; why do you need to navigate around that folder?

Well, signing was failing, so I went into editor settings to see what my debug keystore was set to and make sure it pointed to a valid file. When I clicked the little folder next to it, it didn't show me the already configured file, which was confusing - it made me wonder if maybe the file wasn't there and that was the problem? So, I tried typing assets://keystores/ into the file dialog, which showed me the debug.keystore file, but had the other weird behavior described above.

So, I didn't really need to navigate around there, but it was part of the first thing I tried when attempting to debug why the signing might be failing.

[m4gr3d]

@dsnopek What version of Android is your tablet running? The PKCS12 type is supposed to be supported since api level 1

https://i.sstatic.net/0o6Uq.png

[dsnopek]

It's Android 12:

The information it gives is less detailed than what my phone shows me. :-/

[m4gr3d]

@dsnopek I was able to replicate the issue on a Quest device; I'll investigate further.

[m4gr3d]

@dsnopek I fixed the issue on the Quest. Can you check if your tablet is now able to sign the apk.

[m4gr3d]

The one thing I'm uncertain about: this is also moving custom export templates into advanced settings for all platforms. Should this be included in this PR? Or, a separate PR so that we can discuss that one change with a wider group of contributors?

Good point, I'll move it into a separate PR.

[m4gr3d]

The one thing I'm uncertain about: this is also moving custom export templates into advanced settings for all platforms. Should this be included in this PR? Or, a separate PR so that we can discuss that one change with a wider group of contributors?

Good point, I'll move it into a separate PR.

@dsnopek I move the updates to the custom templates visibility into https://github.com/godotengine/godot/pull/93867.

[Saul2022]

I am getting this error, and can't download and install export template from editor and does not seem to recognize this artifact template.

[m4gr3d]

can't download and install export template from editor and does not seem to recognize this artifact template

That's expected as there's no export template on the server for custom builds. We need to wait until there's an official build with this change for the export template download to work.

In the meantime, you can use the custom_template options to generate a build:

• Unzip the latest export template
• Push the android_debug.apk and android_release.apk binaries to your device
• Configure the custom_template options to point to the binaries you pushed on the device
• Export the project!

If you've already followed those steps and still run into issues, please provide the device logcat so we can investigate the issue.

[Saul2022]

If you've already followed those steps and still run into issues, please provide the device logcat so we can investigate the issue.

I meam that's what i did, the thing is more of the signed error and not sure how to get the error logs cuz it does not let me install or open the aabb

[akien-mga]

Did a cursory code review, it looks good to me overall. Didn't review the refactoring changes of the Android export itself in detail, will need testing on PC to make sure that there's no regression.

My main concern is with the addition of the apksig library, which seems to be a very big dependency (most of the 30k locs of this PR), and a cryptographic one so subject to the likely need for frequent updates to address security vulnerabilities.

Is this a library we really need to vendor, or is it one we could use directly from the Android SDK and link statically? So we're at least always up-to-date with what Google decides to ship in its SDK.

If it's not a SDK library and there's no option but to vendor it, is it something we can trim down to only the parts we absolutely need?

[m4gr3d]

My main concern is with the addition of the apksig library, which seems to be a very big dependency (most of the 30k locs of this PR), and a cryptographic one so subject to the likely need for frequent updates to address security vulnerabilities.

Is this a library we really need to vendor, or is it one we could use directly from the Android SDK and link statically? So we're at least always up-to-date with what Google decides to ship in its SDK.

@akien-mga This is a library we need to vendor as it's not exposed via the Android SDK. The code in this PR was directly taken from the Android repo and the only other form of the library is the apksigner tool which is not available on Android devices.

In terms of maintenance, based on the library directory history, I'd expect a yearly update should be sufficient to pick up the latest updates and bug fixes.

If it's not a SDK library and there's no option but to vendor it, is it something we can trim down to only the parts we absolutely need?

I took a look and the library seems as minimal as it can be. The primary classes we are using are com.android.apksig.ApkVerifier and com.android.apksig.ApkSigner, and both of those have dependencies spanning the entire package, so no options to trim it down.

When compiled, this PR seems to grow the Android build template by ~4mb (66.7mb for android-template-arm64 vs 62.9mb); one option to remove the impact would be to move the apksig library to the Android editor code only since it's only likely to be used by the Android editor builds.

[akien-mga]

When compiled, this PR seems to grow the Android build template by ~4mb (66.7mb for android-template-arm64 vs 62.9mb); one option to remove the impact would be to move the apksig library to the Android editor code only since it's only likely to be used by the Android editor builds.

Yeah I think if we can't do with a smaller impact, then it should at least be editor-only as the size of the export templates is quite sensitive.

[m4gr3d]

When compiled, this PR seems to grow the Android build template by ~4mb (66.7mb for android-template-arm64 vs 62.9mb); one option to remove the impact would be to move the apksig library to the Android editor code only since it's only likely to be used by the Android editor builds.

Yeah I think if we can't do with a smaller impact, then it should at least be editor-only as the size of the export templates is quite sensitive.

Done!

[akien-mga]

Thanks!

[Loki76222]

I tried it worked but it doesn't supporting gradle build........?????? Why????

[m4gr3d]

I tried it worked but it doesn't supporting gradle build........?????? Why????

@Loki76222 We're still investigating how to support gradle builds for the Android editor.

[Loki76222]

In android editor it's not possible to drag and drop from the 'files system'.... please solve it....

[Sai-On-Git]

This PR addresses one of the known limitations of the Godot Android Editor by adding support for generating and exporting Android APK and other platform binaries straight from the Android Editor!

• 1st commit: Add support for the Android-specific assets path prefix

  • This is used to store and access the default debug.keystore used for signing the Android debug builds
  • See here for a full description of the change.

• 2nd commit: Enable support for platform binaries generation:

  • Enables Android apk signing (and verification) using the apksig library from https://android.googlesource.com/platform/tools/apksig/+/ac5cbb07d87cc342fcf07715857a812305d69888

  • Updates export logic to enable platform exports from the Android editor for the following platforms:

    • Android APK exports

      • Note: Only Android legacy builds are supported. Android gradle builds (and AAB exports) are not supported at this point in time.

    • Linux exports

    • Web exports

    • Unsigned Windows exports

    • MacOS exports without notarization and with ad-hoc signature

    • iOS project only exports (limitation for all non-macOS platforms)

Closes https://github.com/godotengine/internal-team-priorities/issues/88 Supersedes #89412 Addresses feedback from #73130

Screen_Recording_20240623_163518.mp4

ARE GRADLE BUILDS(AND AAB BUILDS) SUPPORTED NOW ON GODOT ANDROID EDIOTR, OR THE LEGACY BUILDS CAN ALSO RUN WITH ANDROID 15 TOO?

[Sai-On-Git]

I tried it worked but it doesn't supporting gradle build........?????? Why????

@Loki76222 We're still investigating how to support gradle builds for the Android editor.

SUPPORT ADDED ?

[akien-mga]

Please don't ask questions that were already answered, even more so in an already merged pull request. Please also avoid use CAPS LOCK in your messages, this is often perceived on the Internet as the equivalent of shouting :)

@m4gr3d Probably worth opening an issue about Gradle build support in the Android editor so that interested users can follow it for updates.

[dsnopek]

We have this draft PR where some of the work is happening: https://github.com/godotengine/godot/pull/104069

However, having an issue to point to would be nice, since that PR may end up getting superseded, or the whole process may end up being a series of PRs, and so on