Android 64 bits ARM build config

[danielweck]

Android launcher app: in feature/arm64 branch: https://github.com/readium/readium-sdk/commits/feature/arm64 https://github.com/readium/readium-sdk/commit/3bc240fe340bcef043c683c0314447e473583c07

OpenSSL: in feature/arm64 branch: https://github.com/readium/openssl-android/commits/feature/arm64 https://github.com/readium/openssl-android/commit/3c01400f78ba1e7bb29e51832f530959534e9664 Once compiled, copy static lib to: https://github.com/readium/readium-sdk/tree/develop/ePub3/ThirdParty/openssl-android/lib

ICU4C: TODO: https://github.com/readium/readium-sdk/blob/develop/ePub3/ThirdParty/icu4c/lib/README.md https://github.com/membase/icu4c Once compiled, copy static lib to: https://github.com/readium/readium-sdk/tree/develop/ePub3/ThirdParty/icu4c/lib

[danielweck]

Could this help for ICU? https://github.com/ericwlange/hemroid/tree/master/packages/icu https://github.com/android/platform_external_icu4c

[naveenrajproject]

Checked out feature/arm64 branch and run it in Android Studio 2.2 preview.

"Already compiled icu4c, openssl and boost for arm64 and x86_64. Copied those static libs inside for icu4c: ePub3/ThirdParty/icu4c/lib/arm64-v8a/libicuuc.a, libicudata.a, libicui18n.a, libicuio.a And similarly for openssl: ePub3/ThirdParty/openssl-android/lib/arm64-v8a/libcrypto.a, libssl.a And for boost: ePub3/ThirdParty/boost/lib/arm64-v8a/libboost_regex.a"

After build, Got below Error:

/.../readium-sdk-arm64/ePub3/utilities/CPUCacheUtils.c Error:(31) undefined reference to epub_sys_cache_flush' Error:(28) undefined reference toepub_sys_cache_invalidate' Error:error: ld returned 1 exit status make: *** [/.../readium-sdk-arm64/Platform/Android/epub3/obj/local/arm64-v8a/libepub3.so] Error 1 :epub3:buildMk FAILED Error:Execution failed for task ':epub3:buildMk'.

Process 'command '/.../.../Android-SDK/ndk-bundle/ndk-build'' finished with non-zero exit value 2

It's same error which i have posted here https://github.com/readium/SDKLauncher-Android/issues/96 (2 days ago)

I think, It is missing the file similar as ePub3/utilities/CPUCacheUtils_arm.S

however I tried to compiled by adding "|| defined(_ aarch64 _)" in CPUCacheUtils_arm.S file then it's throws

[arm64-v8a] Compile : epub3 <= CPUCacheUtils_arm.S ././../../../ePub3/utilities/CPUCacheUtils_arm.S: Assembler messages: Error:Error: operand 1 should be an integer register -- mov r3,#0' Error:Error: operand 1 should be an integer register --mov r12,#0x80000000' Error:Error: unknown mnemonic swi' --swi #0x80' Error:Error: unknown mnemonic bx' --bx lr' Error:Error: operand 1 should be an integer register -- mov r3,#1' Error:Error: operand 1 should be an integer register --mov r12,#0x80000000' Error:Error: unknown mnemonic swi' --swi #0x80' Error:Error: unknown mnemonic bx' --bx lr'

[danielweck]

Well, unless ENABLE_SYS_CACHE_FLUSH is defined, the "CPU cache util" stuff should not be included in the build: https://github.com/readium/readium-sdk/blob/develop/ePub3/utilities/byte_buffer.cpp#L27

#ifdef ENABLE_SYS_CACHE_FLUSH
#include "CPUCacheUtils.h"
#endif //ENABLE_SYS_CACHE_FLUSH
...
...

void ByteBuffer::Clean(unsigned char *ptr, size_t len)
{
    bzero(ptr, len);

#ifdef ENABLE_SYS_CACHE_FLUSH
    epub_sys_cache_flush(ptr, len);
#endif //ENABLE_SYS_CACHE_FLUSH
}

See: https://github.com/readium/readium-sdk/commit/0c6c54fa809feddacd92100155e297733d51d89a

Also see: https://github.com/readium/SDKLauncher-iOS/issues/38#issuecomment-66620982

Ah, I see that the "CPU Cache Utils" are included in the build (they can be removed, as they are unused):

https://github.com/readium/readium-sdk/blob/develop/Platform/Android/epub3/Stable.mk#L242 https://github.com/readium/readium-sdk/blob/develop/Platform/Android/epub3/Experimental.mk#L241

LOCAL_SRC_FILES := \
...
    $(EPUB3_PATH)/utilities/CPUCacheUtils_arm.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils_i386.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils_x64.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils.c \

[danielweck]

@naveenrajproject Boost is not needed anymore in Android (develop branch)

[danielweck]

@naveenrajproject can you please explain how you built ICU4C for arm64-v8a (so we can add the instructions in this issue's description)

[naveenrajproject]

Build process for icu4c 64 bit(arm and x86_64) Reference from : http://wudev.digitaltorque.com/2014/03/building-readium-for-x86-android.html

arm64-v8a_AND_ x86_64.zip

1. Download source code from http://site.icu-project.org/download/57#TOC-ICU4C-Download for your host operating system. As I am using Mac, i used http://download.icu-project.org/files/icu4c/57.1/icu4c-57_1-src.tgz
2. extract it.

STEP 1: configure and build for your own machine (we'll build for Android afterwards). You can run these commands individually or past them into a shell script.

ICU$ mkdir build_icu_osx ICU$ cd build_icu_osx ICU/build_icu_osx$ export BASE_ICU_DIR = the parent ICU dir where you downloaded icu ICU/build_icu_osx$ export ICU_SOURCES=$BASE_ICU_DIR/icu-57.1 ICU/build_icu_osx$ export CPPFLAGS="-O3 -DU_USING_ICU_NAMESPACE=1 -fno-short-enums
-DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0
-DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1
-DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0
-DUCONFIG_NO_REGULAR_EXPRESSIONS=1" ICU/build_icu_osx$ sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no
--enable-strict=no -enable-static --enable-shared=no --enable-tests=no
--enable-samples=no --enable-dyload=no ICU/build_icu_osx$ make -j4 ICU/build_icu_osx$ make install ICU/build_icu_osx$ cd ..

STEP 2: Configure and build for Android. Note that the first two lines here may need to be altered for your system, platform, and NDK (to support 64 bit, we need NDK version more than r10c, recomended latest one). ICU$ mkdir build_icu_android ICU$ cd build_icu_android ICU/build_icu_android$ export NDK_ROOT= NDK full path (/Users/smishra/Developement/Android-NDK) ICU/build_icu_android$ export NDK_TOOLCHAIN_BIN=$NDK_ROOT/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/bin ICU/build_icu_android$ export SDK_ROOT=$NDK_ROOT/platforms/android-21/arch-arm64 ICU/build_icu_android$ export ICU_PATH=path of icu source code. (e.g: /Users/smishra/Readium-Thirdparty/ICU/icu-57.1) ICU/build_icu_android$ export ICU_CROSS=path of host build icu (result of step 1) (e.g:/Users/smishra/Readium-Thirdparty/ICU/build_icu_osx) ICU/build_icu_android$ export ICU_FLAGS="-I$ICU_PATH/source/common/ -I$ICU_PATH/source/tools/tzcode/"

ICU/build_icu_android$ export CPPFLAGS="-fno-exceptions --sysroot=$SDK_ROOT -D__STDC_INT64__ $ICU_FLAGS -I$SDK_ROOT/usr/include/ -I$NDK_ROOT/platforms/android-21/arch-arm64/usr/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.9/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/include"

ICU/build_icu_android$ export LDFLAGS="-lsupc++ -lgnustl_static --sysroot=$SDK_ROOT -L$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/ -Wl,-rpath-link=$NDK_ROOT/platforms/android-21/arch-arm64/usr/lib/" ICU/build_icu_android$ export CC=$NDK_TOOLCHAIN_BIN/aarch64-linux-android-gcc ICU/build_icu_android$ export CXX=$NDK_TOOLCHAIN_BIN/aarch64-linux-android-g++ ICU/build_icu_android$ export RANLIB=$NDK_TOOLCHAIN_BIN/aarch64-linux-android-gcc-ranlib ICU/build_icu_android$ export AR=$NDK_TOOLCHAIN_BIN/aarch64-linux-android-ar ICU/build_icu_android$ export LD=$NDK_TOOLCHAIN_BIN/aarch64-linux-android-ld ICU/build_icu_android$ sh $ICU_PATH/source/configure --host=arm-v8a-linux-android --enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no --with-cross-build=$ICU_CROSS --prefix $PWD/icu_build

OR you can execute the attached script file to build for android, as in step 2

STEP 3: Remove compiling ICU tools from the Makefile. Open the Makefile that the configure call created and search for the word "tools." Remove it from the line setting the SUBDIRS variable.

STEP 4: Build ICU ICU/build_icu_android$ make -j4 ICU/build_icu_android$ make install

[danielweck]

Ah, I see that the "CPU Cache Utils" are included in the build (they can be removed, as they are unused):

https://github.com/readium/readium-sdk/blob/develop/Platform/Android/epub3/Stable.mk#L242 https://github.com/readium/readium-sdk/blob/develop/Platform/Android/epub3/Experimental.mk#L241

LOCAL_SRC_FILES := \
...
    $(EPUB3_PATH)/utilities/CPUCacheUtils_arm.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils_i386.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils_x64.S \
    $(EPUB3_PATH)/utilities/CPUCacheUtils.c \

[danielweck]

Note that I am proposing that we remove the dependency on OpenSSL for Android (one less static lib!) https://github.com/readium/readium-sdk/pull/247

[danielweck]

FYI, CPU Cache Utils asm now removed from Android config (was unused, but was causing issues in 64 bits build) https://github.com/readium/readium-sdk/issues/248

[naveenrajproject]

I don't know much about those assembly code in CPUCacheUtils_arm.S file. So simply I add following lines at bottom of this file. #if defined(aarch64) /* void epub_sys_cache_invalidate(void* start, size_t len) / LABEL(C_FN_NAME(epub_sys_cache_invalidate)) / void epub_sys_cache_flush(void* start, size_t len) */ LABEL(C_FN_NAME(epub_sys_cache_flush)) #endif then it works for arm64-v8a (generated libepub3.so file)

[danielweck]

Related: https://github.com/readium/readium-sdk/pull/240#issuecomment-255785818 (Clang build)

[danielweck]

Well, support for ICU is explicitly disabled in xmlversion.h: https://github.com/readium/readium-sdk/blob/develop/ePub3/ThirdParty/libxml2-android/include/libxml/xmlversion.h#L271 which means that the ICU4c pre-built static libs are useless: https://github.com/readium/readium-sdk/tree/develop/ePub3/ThirdParty/icu4c/lib ..and so are the headers: https://github.com/readium/readium-sdk/tree/develop/ePub3/ThirdParty/icu4c/include/unicode

So, I removed the ICU4c dependencies from my build, and everything compiles, links, and runs fine. One less thing to worry about as I am working my way through segmentation faults in Android :)

[junaidTbox]

@danielweck hi i have cloned from main branch and i know it is old branch you guys are working on development branch. And on development branch there are two flavors x86 an armeabi-v7a. But epub library is not loading on my arm64 and keep crashing. Any updates / solution to fix this issue.

[danielweck]

I haven't tried myself, but in principle you can use the master or develop branches of readium-sdk, edit the Android *.mk files (assuming you are using Gradle "stable"), or the Gradle "experimental" *.gradle file. For example: Platform/Android/epub3/Application.mk:

APP_ABI := arm64-v8a armeabi-v7a x86

Platform/Android/epub3/build_experimental.gradle:

android.productFlavors {
create ("arm64") {
   ndk.with {
   abiFilters.add("arm64-v8a")
   ldFlags.addAll([
     "-L${file("./obj/local/arm64-v8a")}".toString()
   ])
  }
}

[adnanali-indpro]

Error:error: conflicting declaration 'typedef long long unsigned int uint64_t'... getting this error when i am using APP_ABI := arm64-v8a armeabi-v7a

[adnanali-indpro]

@danielweck any idea for the error? (Error:error: conflicting declaration 'typedef long long unsigned int uint64_t'... getting this error when i am using APP_ABI := arm64-v8a armeabi-v7a).. not able to generate .so for arm64.

[danielweck]

I have not personally looked into this issue for a long time, but it seems developers in this conversation thread (like @naveenrajproject ) have succeeded. Perhaps we can get an update from them?

[adnanali-indpro]

@danielweck thanks alot. @naveenrajproject : any idea for the error? (Error:error: conflicting declaration 'typedef long long unsigned int uint64_t'... getting this error when i am using APP_ABI := arm64-v8a armeabi-v7a).. not able to generate .so for arm64.

[adnanali-indpro]

finally i resolved the issue.@ https://stackoverflow.com/questions/32198368/unsigned-long-long-conflict-with-uint64-t helped me. i got the so for 64.

[adnanali-indpro]

its resolved now. i am able to get armeabi-v8a libepub3.so file(64bit).

[rkwright]

@adnanali-indpro Thanks for the great feedback. Flagging as useful FAQ/doc info.

[jdempcy]

its resolved now. i am able to get armeabi-v8a libepub3.so file(64bit).

@adnanali-indpro How were you able to resolve it?

I am running into the same error: conflicting declaration 'typedef long long unsigned int uint64_t'.

I find 8 matches in 5 files for "unsigned long long" and 5 matches in 3 files for "uint64_t". Do I just have to change them all to one or the other for consistency's sake, because in 32-bit they equate to the same but we're getting an error in 64-bit because of a difference between them?

[jdempcy]

I was able to fix that issue by removing the typedef lines in the following files:

• readium-sdk/ePub3/ThirdParty/google-url/src/url_canon_ip.cc
• readium-sdk/UnitTests/catch.hpp

In each of those files, I commented out the following line:

typedef unsigned long long uint64_t;

[jdempcy]

I ran into another error related to paths, which I was able to fix:

:epub3:buildMk
make: Entering directory `/Users/jdempcy/Development/android-cloudshelf-reader/android-cloudshelf-reader/epub3'
[arm64-v8a] Compile        : epub3 <= mkstemp.c
[arm64-v8a] Compile        : epub3 <= zip_add.c

In file included from ././../../android-cloudshelf-reader-sdk/readium-sdk/ePub3/ThirdParty/libzip/zipint.h:39:0,
                 from ././../../android-cloudshelf-reader-sdk/readium-sdk/ePub3/ThirdParty/libzip/zip_add.c:36:
././../../android-cloudshelf-reader-sdk/readium-sdk/ePub3/ThirdParty/libzip/zip.h:37:27: fatal error: ePub3/_config.h: No such file or directory
 #include <ePub3/_config.h>
                           ^
compilation terminated.
make: *** [/Users/jdempcy/Development/android-cloudshelf-reader/android-cloudshelf-reader/epub3/obj/local/arm64-v8a/objs/epub3/./__/__/android-cloudshelf-reader-sdk/readium-sdk/ePub3/ThirdParty/libzip/zip_add.o] Error 1

make: Leaving directory `/Users/jdempcy/Development/android-cloudshelf-reader/android-cloudshelf-reader/epub3'
:epub3:buildMk FAILED

FAILURE: Build failed with an exception.

I fixed it by updating the path like so:

#include <../../android-cloudshelf-reader-sdk/readium-sdk/ePub3/_config.h>

I'm not sure why it was executing from another local path but that fixed it for me.

[jdempcy]

I was actually able to work around the pathing issue without necessitating a change to the C++ code. I believe it was by editing a project.properties file to add the path like so:

android.library.reference.1=../readium-sdk/Platform/Android

It may have been something else, though. Unfortunately, a cursory look through my git history is not revealing the exact change that fixed it. But if anyone else runs into this path error as well, let me know and I can try to find how I fixed it. I believe I was searching for something like "set C++ root path android studio" when I found the solution.

[danielweck]

FYI: https://github.com/readium/readium-sdk/pull/317 => working ReadiumSDK 64bits build, with both latest Gradle (NDK Makefile) and Gradle experimental plugin (NDK DSL). AARs now created with dual 32-64 bits libs, and with ARM+x86, or ARM alone, or x86 alone

[aviashiet]

@adnanali-indpro - Please suggest how you are able to get armeabi-v8a libepub3.so file(64bit). can you share that ?

[danielweck]

You can build this yourself using this Pull Request / branch: https://github.com/readium/readium-sdk/pull/317