DiligentEngine
DiligentEngine copied to clipboard
DiligentGraphics
Publish documentation of command line android build of SDK and Samples
Ideally, users of Android could build from the command line, assuming necessary components are installed from the SDk. It would be great to have the gradle commands documented to build specific projects and also necessary commands install install/run/debug. Possibly related to #41. I've done this for other projects, specifically figured it out for the messy Oculus Android SDK (a monolithic Android SDK/examplewad that cannot be separated). I'll poke around around on this to see if I can tease out the commands from the ./gradlew step to bootstrap and the available commands. This assumes too a few pre-reqs: NDK, android SDK tools, android SDK API versions (what is min needed, what is target), any hardware support needed (I think this is posted somewhere in the root level docs), and Java tools that should be in place (openjdk version).
Android build system is very poorly set up at the moment, but unfortunately we now have no bandwidth to improve it and any help would be greatly appreciated.
Command-line build will also enable running Android build on CI servers as right now it is the only platform that is skipped by CI
Ok, so yeah, that's something I'd like to make sure we can automate. I feel like Android Studio makes some things easier (recommending fixes for missing components), but when it fails, it can be puzzling. In most cases one can get better information from the command line build with gradle. I'll see what I find.
First problem I ran into was related to Error:Could not find com.android.support.constraint:constraint-layout:1.0.1 required by AntTweakBar. I did a search on this in the mvn repositories (it is a support library), and it is not in mvnrepository, oddly, so is probably packaged by the Android Studio version, but not determined what version. Lots of people have hit this problem on other projects. The best thing to do is change all versions of this support lib from 1.0.1 to 1.1.0, as that is in the mvnrepository.
I'll propose a fix for the above, on to the next issue: Task :Common:generateJsonModelDebug fails with a cmake error (probably NDK related). Need to investigate.
Error:Could not find com.android.support.constraint:constraint-layout:1.0.1
Where is it specified? Can't find it in Gradle files or in the project generally. BTW, do you have the latest code? I recently made few changes/fixes on Android
Oh, let me pull the latest. All of these constraint things are in the samples:
./AntTweakBar/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.1.0'
./Atmosphere/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial01_HelloTriangle/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial02_Cube/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial03_Texturing/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial04_Instancing/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial05_TextureArray/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial06_Multithreading/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial07_GeometryShader/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial08_Tessellation/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial09_Quads/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial10_DataStreaming/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
./Tutorial11_ResourceUpdates/build.gradle: implementation 'com.android.support.constraint:constraint-layout:1.0.1'
Yeah, I double checked the history and these were indeed removed recently https://github.com/DiligentGraphics/DiligentSamples/commit/847e1c33699bc46adc57799022272eeddf88a864
Ok, that was a good update, fixed that last problem. Hit a new one:
Execution failed for task ':AntTweakBar:processDebugResources'.
> Android resource linking failed
/.gradle/caches/transforms-2/files-2.1/b229f0a2317aecc488ca7e6d10403f98/res/values-v28/values-v28.xml:9:5-12:13: AAPT: error: resource android:attr/dialogCornerRadius not found.
/dev/repos/DiligentEngine/DiligentSamples/Android/build/AntTweakBar/intermediates/incremental/mergeDebugResources/merged.dir/values-v28/values-v28.xml:11: AAPT: error: resource android:attr/dialogCornerRadius not found.
/.gradle/caches/transforms-2/files-2.1/afd4068bb0a5baef6d45864dae65a31e/res/values/values.xml:89:5-125:25: AAPT: error: resource android:attr/fontVariationSettings not found.
/.gradle/caches/transforms-2/files-2.1/afd4068bb0a5baef6d45864dae65a31e/res/values/values.xml:89:5-125:25: AAPT: error: resource android:attr/ttcIndex not found.
Probably something I'm missing in my support libraries or SDK version. Looking into that.
I'll revisit this later in the week (SDK updating right now), and I have to jump onto something else.
Thanks for looking into this. TBH, I have no idea what these errors mean. Looks like none of these attributes are ever specified by any of build files... Also, sometimes incremental build does not handle the file changes properly, so it may be useful to delete all intermediate files.
As an aside, since I am only building with gradlew + a standalone Android SDK, I have to go through this trial and error. The Android Studio will usually suggest/recommend fixes that will often work. However, I have to sort out the meaning of the errors. There are a lot of moving parts (platform/os, SDK tool version, support libs, ndk tool version, cmake version, and probably a few other things I forgot). I'll keep digging. Those errors are probably some library version or tools version that I am missing.
You are right. Android studio indeed suggests installing a number of components. Maybe if you install it and open the project, you will know what components are missing.
So as far as this error:
Execution failed for task ':AntTweakBar:processDebugResources'.
> Android resource linking failed
/.gradle/caches/transforms-2/files-2.1/b229f0a2317aecc488ca7e6d10403f98/res/values-v28/values-v28.xml:9:5-12:13: AAPT: error: resource android:attr/dialogCornerRadius not found.
/dev/repos/DiligentEngine/DiligentSamples/Android/build/AntTweakBar/intermediates/incremental/mergeDebugResources/merged.dir/values-v28/values-v28.xml:11: AAPT: error: resource android:attr/dialogCornerRadius not found.
/.gradle/caches/transforms-2/files-2.1/afd4068bb0a5baef6d45864dae65a31e/res/values/values.xml:89:5-125:25: AAPT: error: resource android:attr/fontVariationSettings not found.
/.gradle/caches/transforms-2/files-2.1/afd4068bb0a5baef6d45864dae65a31e/res/values/values.xml:89:5-125:25: AAPT: error: resource android:attr/ttcIndex not found.
it appears that this can be fixed by adding:
compileSdkVersion 28
to the build.gradle in the offending project AntTweakBar.
I'm getting another error after fixing the last one in the cmake section of building AntTweakBar
gnustl_static is no longer supported. Please switch to either c++_shared
or c++_static. See
https://developer.android.com/ndk/guides/cpp-support.html for more
information.
So this above issue is a problem with the changes to the NDK. @DiligentGraphics unfortunately for maintainers, SDK tools, OS versions, and support tools provided by the Android SDK toolchains don't stand still, and stuff that might build perfectly in your configuration might not on another system due to the variety of differences that might exist. It is sometims necessary to save off versions of tools in case things aren't being archived by Google (I think for the most part they do archive things, but they've moved away from the standalone SDK tools to Android Studio, which pushes the burden of maintaining Studio also. The ideal strategy that I use is to lock down configuration and make sure that I document what I'm using, such that anyone else picking up my project will be able to use the same versions and be guaranteed success in the build. Use something different and the guarantees are completely thrown off. I think that's where this is at, unless you are telling me that you've got other people building on other systems for you to validate and that maybe it's my configuration that's crazy.
In any case, I'll keep plugging away here until I nail down a configuration that works. My team is sufficiently motivated to get the samples working and making builds reproducible.
So the above C++ / cmake issue mentioned is down in the guts of the NDK. Probably with NDK 19 or sometime earlier, they no longer support gnustl_static. To fix this requires a change to:
Common/NativeApp/Android/build.gradle
and change the line for the cmake and just don't specify the STL
externalNativeBuild {
cmake {
arguments '-DANDROID_PLATFORM=android-21',
'-DANDROID_TOOLCHAIN=clang', '-DENABLE_TESTS=TRUE'
}
}
That fixes the other bug I hit.
In the midst of building [19/65] Linking CXX static library DiligentTools/TextureLoader/libTextureLoader.a
I get about 20 errors, starting with:
DiligentEngine/DiligentSamples/External/TwBarLib/src/TwGraphImpl.cpp:135:32: error: reference to type 'const Diligent::TextureData' could not bind to an rvalue of type 'Diligent::TextureData *'
_Dev->CreateTexture( desc, &TexData, _Tex );
^~~~~~~~
I'm going to try on an older setup with an older sdk / ndk. These problems maybe are due to the change away from the gnustl-static.
Do you get this error after removing -DANDROID_STL=c++_static? The error seems to be a compiler error, which shouldn't be related to c++ runtime library controlled by DANDROID_STL.
In Android Studio, the build succeeds with the following build settings:
cmake {
arguments '-DANDROID_TOOLCHAIN=clang', '-DENABLE_TESTS=TRUE'
}
I can't understand why it causes so many errors in a standalone toolchain.
BTW, are you able to build the teapot sample from android ndk samples (which is where I copied the settings from)?
So not sure as far as why the standalone toolchain is having an issue, but as I said, I'm on Mac OS X, exactly same errors with Android Studio as with building from the command line. I'm using the same toolchain, just that Android Studio manages the tools that get installed. My belief is these Android build issues are 95% of the time configuration issues, not necessarily a bug in the settings per se or code issues. In other words, something in my environment, tools versions, or other binary that might be in my path that doesn't match the settings or api version.
In terms of the gnustl static issue, described in this error message below, that's just an NDK version issue. If you are still on an older NDK that won't affect your build. So the first question is, what NDK version do you have installed? I'll separately confirm that the teapot builds.
gnustl_static is no longer supported. Please switch to either c++_shared or c++_static. See https://developer.android.com/ndk/guides/cpp-support.html for more information.
gnustl_static is no longer supported. Please switch to either c++_shared or c++_static. See https://developer.android.com/ndk/guides/cpp-support.html for more information.
I do use c++_static, but you said it generates an error. I'm little confused now.
@DiligentGraphics so if you don't mind, can you state which version the Android NDK you are using?
A second point, that I forgot was that this project has submodules, and to keep everything working together, I have to remember to do the git submodule update step.
I'm setting up a windows system with a pure command line set up (Google has un-hidden the Android Command Line SDK tools). I'll keep the mac with the Android studio.
About to retest
So after resetting my workspace back to a clean state and doing a submodule update --recursive, I get an error I've seen before on other projects.
Ran a ./gradlew after setting the gradlew to be executable.
* What went wrong:
A problem occurred configuring project ':AntTweakBar'.
> No toolchains found in the NDK toolchains folder for ABI with prefix: mips64el-linux-android
So on a non-linux platform, we aren't going to have mips64el-linux-android cross compiler under the toolchains. That affects windows and mac. The solutions are numerous, but I'll refer to the Google Filament issue as somewhat authoratative:
https://github.com/google/filament/issues/15
The second to last solution worked for me.
Continuing to build.
I currently have NDK 19.1
I can try to update it to 20.0
I can successfully build everything on Mac and Linux with Android Studio 3.4.1. It used to work on Windows too, but it was unusably slow and I haven't tried it for a while.
running: gradlew build
hits the same problem seen in comment: https://github.com/DiligentGraphics/DiligentEngine/issues/42#issuecomment-494212422
Before I repeat the entire sequence of errors that I found previously, what tag or branch should I be building from, if not tag v2.4.b?