Port to Android

[seven332]

I'm porting this project to Android and it works fine. I can pull request if you need it. The fork is HERE. The Android Studio project is HERE.

But it crashes when creating an archive in 7z format. I found out that a new thread (maybe more than one thread) was created from native code when creating an archive in 7z format. In Android, FindClass only uses system class loader in this kind of thread. So the application class like net.sf.sevenzipjbinding.*** can't be found. I found an article about this problem. HERE. The article suggests some solutions. But I am not familiar with this project, it is hard for me to do the job. Maybe you can solve the problem.

[borisbrodski]

Hi! That's awesome!

Did you managed to run tests on Android devices or VM?

I will take a look of this problem. But this may take a while. Do you have some concrete questions?

(Check migrate-to-15.09-try2 branch. It actually almost complete migration to 7zip 15.09)

[seven332]

I did some simple tests on my phone. Create archive and extract it. Zip, Tar, GZip, BZIP2 passed, 7z failed. And extracting 7z archive works find. I think the problem is what I said before. I will do more test soon.

One question. How to create a archive with password?

I have checked the migrate-to-15.09-try2 branch. You can check my commit here. https://github.com/seven332/sevenzipjbinding/commits/migrate-to-15.09-try2

[seven332]

I think I solved the problem. 7z archive creating works now on Android. But it crashed on Google Android simulator API 15 x86. I can't figure out why it crashed. And it works on the other simulator like API 14, API 23 as well as my phone Nexus 5 cm13. Maybe the API 15 simulator itself is broken. I don't own any phone with Android 4.0.3 namely API 15. So I'm not sure.

I will pull request after more tests.

[borisbrodski]

Cool! I reviewed your branch. Great work :+1:. Thank you! I added some line comments, I hope you get notified.

Also some important questions remain:

• Do we use the same binaries on Android and non-Android platforms? (For example, do Linux ARM binary run on ARM-based Android device?)
• How we load the native library corresponding to the running platform (32/64, ARM/Intel/...)
• Can we build Android binaries headless?
• How can we run the entire test suite on an Android device?

[borisbrodski]

Oh, I think, the App-Store will choose the correct version of the App depending on the platform, right?

[seven332]

1. They are different. We use Android NDK to build native library on Android.
2. Android system will select the native library when user install the app.
3. We can build Android binaries with one command line.
4. We need Android Studio or a gradle project to run test on an Android device. I ported your JavaTest to Android and ran it on my Android device today. I got a really really weird native crash.

[seven332]

The is the log https://gist.github.com/seven332/77ab8ee5f0f8adbc3c4bfe75a20c5341

[seven332]

Here is the log from ndk-stack

********** Crash dump: **********
Build fingerprint: 'google/hammerhead/hammerhead:6.0.1/MOB30D/2704746:user/release-keys'
pid: 27393, tid: 27421, name: roidJUnitRunner  >>> com.hippo.a7zip.test <<<
signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
Stack frame 05-20 02:28:50.509  1162  1162 F DEBUG   :     #00 pc 00042188  /system/lib/libc.so (tgkill+12)
Stack frame 05-20 02:28:50.509  1162  1162 F DEBUG   :     #01 pc 0003fd95  /system/lib/libc.so (pthread_kill+32)
Stack frame 05-20 02:28:50.509  1162  1162 F DEBUG   :     #02 pc 0001c497  /system/lib/libc.so (raise+10)
Stack frame 05-20 02:28:50.509  1162  1162 F DEBUG   :     #03 pc 00019649  /system/lib/libc.so (__libc_android_abort+34)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #04 pc 00017500  /system/lib/libc.so (abort+4)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #05 pc 003319c5  /system/lib/libart.so (_ZN3art7Runtime5AbortEv+228)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #06 pc 000f462b  /system/lib/libart.so (_ZN3art10LogMessageD2Ev+2226)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #07 pc 0025aa7f  /system/lib/libart.so (_ZN3art9JavaVMExt8JniAbortEPKcS2_+1550)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #08 pc 0025ae2d  /system/lib/libart.so (_ZN3art9JavaVMExt9JniAbortVEPKcS2_St9__va_list+64)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #09 pc 000fd201  /system/lib/libart.so (_ZN3art11ScopedCheck6AbortFEPKcz+32)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #10 pc 000fd38f  /system/lib/libart.so (_ZN3art11ScopedCheck17CheckMethodAndSigERNS_18ScopedObjectAccessEP8_jobjectP7_jclassP10_jmethodIDNS_9Primitive4TypeENS_10InvokeTypeE+190)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #11 pc 00110bbf  /system/lib/libart.so (_ZN3art8CheckJNI11CallMethodVEPKcP7_JNIEnvP8_jobjectP7_jclassP10_jmethodIDSt9__va_listNS_9Primitive4TypeENS_10InvokeTypeE+562)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #12 pc 001124db  /system/lib/libart.so (_ZN3art8CheckJNI17CallObjectMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+34)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #13 pc 000c1561  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZN7_JNIEnv16CallObjectMethodEP8_jobjectP10_jmethodIDz+76): Routine _JNIEnv::CallObjectMethod(_jobject*, _jmethodID*, ...) at E:/Android/android-sdk/ndk-bundle/build//../platforms/android-9/arch-arm/usr/include/jni.h:636 (discriminator 2)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #14 pc 000d502d  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZN3jni18IOutCreateCallback18getItemInformationEP7_JNIEnvP8_jobjectiS4_+92): Routine jni::IOutCreateCallback::getItemInformation(_JNIEnv*, _jobject*, int, _jobject*) at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/jbinding-cpp/JavaStatInfos/JavaPackageSevenZip.h:129 (discriminator 2)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #15 pc 000d4eb1  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZN30CPPToJavaArchiveUpdateCallback18getOrUpdateOutItemER14JNIEnvInstancei+140): Routine CPPToJavaArchiveUpdateCallback::getOrUpdateOutItem(JNIEnvInstance&, int) at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/jbinding-cpp/CPPToJava/CPPToJavaArchiveUpdateCallback.cpp:31 (discriminator 1)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #16 pc 000d511b  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZN30CPPToJavaArchiveUpdateCallback17GetUpdateItemInfoEjPiS0_Pj+82): Routine CPPToJavaArchiveUpdateCallback::GetUpdateItemInfo(unsigned int, int*, int*, unsigned int*) at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/jbinding-cpp/CPPToJava/CPPToJavaArchiveUpdateCallback.cpp:68
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #17 pc 000d5889  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZTv0_n36_N30CPPToJavaArchiveUpdateCallback17GetUpdateItemInfoEjPiS0_Pj+64): Routine virtual thunk to CPPToJavaArchiveUpdateCallback::GetUpdateItemInfo(unsigned int, int*, int*, unsigned int*) at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/jbinding-cpp/CPPToJava/CPPToJavaArchiveUpdateCallback.cpp:60
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #18 pc 0018c6fd  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (_ZN8NArchive4NZip8CHandler11UpdateItemsEP20ISequentialOutStreamjP22IArchiveUpdateCallback+236): Routine NArchive::NZip::CHandler::UpdateItems(ISequentialOutStream*, unsigned int, IArchiveUpdateCallback*) at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/p7zip/CPP/7zip/Archive/Zip/ZipHandlerOut.cpp:90
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #19 pc 000cf1f7  /data/app/com.hippo.a7zip.test-2/lib/arm/lib7z.so (Java_net_sf_sevenzipjbinding_impl_OutArchiveImpl_nativeUpdateItems+790): Routine Java_net_sf_sevenzipjbinding_impl_OutArchiveImpl_nativeUpdateItems at E:/github/a7zip/library/src/main/jni/sevenzipjbinding/jbinding-cpp/JavaToCPP/JavaToCPPOutArchiveImpl.cpp:80 (discriminator 3)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #20 pc 0028e4c9  /data/app/com.hippo.a7zip.test-2/oat/arm/base.odex (offset 0x174000) (void net.sf.sevenzipjbinding.impl.OutArchiveImpl.nativeUpdateItems(net.sf.sevenzipjbinding.ISequentialOutStream, int, java.lang.Object)+132)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #21 pc 0028e383  /data/app/com.hippo.a7zip.test-2/oat/arm/base.odex (offset 0x174000) (void net.sf.sevenzipjbinding.impl.OutArchiveImpl.doUpdateItems(net.sf.sevenzipjbinding.ISequentialOutStream, int, net.sf.sevenzipjbinding.IOutCreateCallback)+94)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #22 pc 0028e7b5  /data/app/com.hippo.a7zip.test-2/oat/arm/base.odex (offset 0x174000) (void net.sf.sevenzipjbinding.impl.OutArchiveImpl.createArchive(net.sf.sevenzipjbinding.ISequentialOutStream, int, net.sf.sevenzipjbinding.IOutCreateCallback)+72)
Stack frame 05-20 02:28:50.510  1162  1162 F DEBUG   :     #23 pc 002b7a1d  /data/app/com.hippo.a7zip.test-2/oat/arm/base.odex (offset 0x174000) (void net.sf.sevenzipjbinding.impl.OutArchiveZipImpl.createArchive(net.sf.sevenzipjbinding.IOutStream, int, net.sf.sevenzipjbinding.IOutCreateCallback)+64)
Stack frame 05-20 02:28:50.511  1162  1162 F DEBUG   :     #24 pc 00253217  /data/app/com.hippo.a7zip.test-2/oat/arm/base.odex (offset 0x174000) (void net.sf.sevenzipjbinding.junit.compression.StandaloneCompressZipTest.testCompressionZip()+746)
Stack frame 05-20 02:28:50.511  1162  1162 F DEBUG   :     #25 pc 000e6361  /system/lib/libart.so (art_quick_invoke_stub_internal+64)
Stack frame 05-20 02:28:50.511  1162  1162 F DEBUG   :     #26 pc 004028a5  /system/lib/libart.so (art_quick_invoke_stub+188)
Stack frame 05-20 02:28:50.511  1162  1162 F DEBUG   :     #27 pc 00101dc4  [stack:27421]

[seven332]

It says

JNI DETECTED ERROR IN APPLICATION: the return type of CallObjectMethodV does not match net.sf.sevenzipjbinding.IOutItemBase net.sf.sevenzipjbinding.IOutCreateCallback.getItemInformation(int, net.sf.sevenzipjbinding.impl.OutItemFactory)

But the return type is right.

[seven332]

Sorry for the long log

[borisbrodski]

np, Most of the time I spend on 7-Zip-JBinding was fixing such weird bugs. JNI crashed due to number of reasons:

• simple memory corruption
• forgotten exception check after method or constructor call
• missmatch in types (this shouldn't happen, if using JavaStaticInfo.h)
• Multithreading problems
• Compiler bugs (had already two of them)
• 7-Zip bugs

Some thoughts:

• Is it extraction only or compression only problem? (or both)
• Is it some specific test, that causes the crash?
• Is it one of XxxMultithreaded tests? (You can set number of threads to 1 to test single threaded case, there are no single constant for that, probably two or three, sorry)

I will install Android studio and try to help you. But this may take some time (not in May)

[seven332]

Thanks for you advice. All tests passed. It really takes a long time to pass all test. Three hours and a half on my phone.

The native crash I said before is because of the java.reflect.Proxy. It seems Proxy doesn't works on Android through JNI. I commented it out and the test passed. But I couldn't find any thing about it on Android official documentation or any one's blog.

Then I got another native crash like this.

JNI ERROR (app bug): local reference table overflow (max=512)

Local references is precious resources on Android. I added more DeleteLocalRef to avoid the crash.

[borisbrodski]

That's great news! Cool!

[seven332]

I think 7-Zip-JBinding is OK for Android now.

I intend to create a folder with name "Android" in the project root. It contains a gradle project which can build the library file for Android. Is it OK ?

I did some modifications in test source files to make sure Android devices can pass the tests. For example, Android need an inconvenient way to load test data, and the Proxy problem as I said before. I can't make the test works both on Android and PC. But I think I can create some separate files for Android only to solve it.

Is the java.reflect.Proxy necessary? If it goes away, I will create less separate files.

[borisbrodski]

Thank you very much!

I will look into it this week. I have to install Android Studio and try it for myself.

[borisbrodski]

SORRY, it took way more time as I thought it would. It looks, like I have to reinstall my Ubuntu first. (I have old 32 bit version running, it's a pain to find binaries for it).

I will be on a vacation in august, so may be I will have more time to play with Android and stuff.

Are you actually already using your build in some project of yours?

Thank you for your patience!

[seven332]

I was planning to create a app with browsing files in archives support, then I found you project. It is still a plan now.

Android Studio is not necessary, gradle is enough to build and test. Install android sdk and ndk, clone this project, connect you Android phone to PC, run

./gradlew :library:installDebugAndroidTest
adb shell am instrument -w com.hippo.a7zip.test/android.support.test.runner.AndroidJUnitRunner

[vt-ombalakumar]

Hi seven332, how to integrate your project seven332/a7zip in my app..

[seven332]

Let jitpack help you. Follow the instructions here. https://jitpack.io/#seven332/a7zip/1fcf6b889a

It's not a formal release. I don't give any warranty.

[vt-ombalakumar]

wow, thanks for the fast reply. I will try this.. Thank you

[borisbrodski]

Cool!

I'm also working on adding Android support (based on your contribution) to the 7-Zip-JBinding, but it will take time due to my poor Android skills...

[4ntoine]

Any update on this?