google
libc++_shared.so from library that uses Oboe is clashing with another library
Android version(s): N/A (didn't get that far) Android device(s): N/A (as above) Oboe version: 1.5.0 App name used for testing: A closed-source app, sorry
Short description This is a build issue.
I've made an Android library which uses Oboe (from prefab), called MidiPlayer. As advised in GettingStarted.md, I'm using libc++_shared.so.
I also have an Android app. In the app, I include both MidiPlayer and a totally unrelated 3rd party library. I get the following error:
More than one file was found with OS independent path 'lib/arm64-v8a/libc++_shared.so'. If you are using jniLibs and CMake IMPORTED targets, see https://developer.android.com/studio/preview/features#automatic_packaging_of_prebuilt_dependencies_used_by_cmake
Unhelpfully, that link is wrong. But I think it's supposed to point here, however this is irrelevant (I don't have a jniLibs directory).
The issue is that by following the advice in GettingStarted.md, I end up with a copy of libc++_shared.so in the package. Specifically, when I look in my .gradle folder I see a folder called jetified-midiplayer-<version> which contains a jni directory, and for each platform folder it contains libc++_shared.so. This appears to be clashing with other Android libraries which do the same thing. How am I supposed to do this? Should I just use the c++_static Android STL? I figure that's bad form because it increases the size of my app. It also goes against the advice in GettingStarted.md.
Detailed description
Disclaimer: I'm not experienced with C or C++. I am just trying to follow instructions.
My library (MidiPlayer) has a build.gradle that looks something like this:
apply plugin: 'com.android.library'
apply plugin: 'kotlin-android'
apply plugin: 'maven-publish'
android {
compileSdkVersion 30
buildToolsVersion "30.0.0"
defaultConfig {
minSdkVersion 23
targetSdkVersion 30
versionCode project.versionCode
versionName project.versionName
...
externalNativeBuild {
cmake {
arguments "-DANDROID_STL=c++_shared"
}
}
}
externalNativeBuild {
cmake {
path "CMakeLists.txt"
version "3.10.2"
}
}
buildFeatures {
prefab true
}
}
dependencies {
...
implementation 'com.google.oboe:oboe:1.5.0'
}
My CMakeLists.txt looks like this:
cmake_minimum_required(VERSION 3.4.1)
add_library (nativeMidiSynth SHARED path/to/source1.cpp path/to/source2.cpp)
find_package (oboe REQUIRED CONFIG)
target_link_libraries(nativeMidiSynth log oboe::oboe)
I then publish this Android library to a maven repository. Over in my app (separate codebase), I pull this and the clashing library (pdfium, a dependency of android-pdf-viewer) in with:
dependencies {
implementation 'com.github.barteksc:android-pdf-viewer:3.2.0-beta.1'
implementation "com.example:midiplayer:0.1.0"
}
Then, when building I see the error mentioned above. By searching through my .gradle directory I found that these are the two packages that contain libc++_shared.so. For example, my MidiPlayer's jni\arm64-v8a directory (in C:\Users\chris\.gradle\caches\transforms-2\files-2.1\cb0f0382d7f01637cf539290e3e3d990\jetified-midiplayer-1.3.1) contains 3 files:
-
libc++_shared.so -
libnativeMidiSynth.so- my native library which uses oboe -
liboboe.so
pdfium's jni\arm64-v8a directory (in C:\Users\chris\.gradle\caches\transforms-2\files-2.1\d99f523756be4de3c5d148ea12d9cbab\jetified-pdfium-android-1.9.0) contains these files:
-
libc++_shared.so -
libjniPdfium.so -
libmodft2.so -
libmodpdfium.so -
libmodpng.so
So how do I, you know, share this library? Why does it cause a clash?
From googling around, I found a workaround which is to add the following to my app's build.gradle:
packagingOptions {
pickFirst 'lib/armeabi-v7a/libc++_shared.so'
pickFirst 'lib/arm64-v8a/libc++_shared.so'
pickFirst 'lib/x86/libc++_shared.so'
pickFirst 'lib/x86_64/libc++_shared.so'
}
As far as I understand it, this will force the packager to ignore the second copy of libc++_shared.so that it sees. However, this feels dangerous - what if they are different versions of C++? I don't know if that's a real problem. Is this workaround an acceptable solution? Why isn't it documented?
Your "workaround" is correct. Gradle was given two conflicting inputs and has no way to resolve them automatically. The two libraries named "libc++_shared.so" might be compatible or they might not be; there's no way for the build system to know, so the user needs to resolve the conflict manually. Prior versions of AGP did not diagnose this error and one was chosen arbitrarily.
That said, the dependencies could have been built differently so that it wouldn't need to be the user's problem. https://android.googlesource.com/platform/ndk/+/master/docs/user/middleware_vendors.md#for-java-middleware-with-jni-libraries explains a bit about this (I just sent a change to migrate that to dac, so will be more visible soon). If I'm understanding your report correctly, you can't fix this on your end: libpdfium has forced libc++_shared.so on your app, as has oboe.
As for why this was only a problem when building your app and not when building libnativeMidiSynth: because oboe is distributed as an AAR, AGP (or rather the tool that handles the import of the dependency) actually does have more information for that library because the AAR includes information about the exact version of libc++ that was used. It can check that the version libnativeMidiSynth is using is compatible with the one used by liboboe and as such there's no issue. libpdfium is not distributed in this manner (it seems), and neither is libnativeMidiSynth, so when building your app you need to manually resolve the conflict. Unfortunately packaging libnativeMidiSynth in this way won't solve the problem if libpdfium doesn't also do that.
tl;dr I think your "workaround" is the best solution you have unless libpdfium can be repackaged as an AAR.
Unhelpfully, that link is wrong.
Fixed in later versions of AGP, but yeah, annoying.
However, this feels dangerous - what if they are different versions of C++? I don't know if that's a real problem.
You're right, getting this wrong can introduce bugs. That's why we emit errors for this case now. If libpdfium was built with the same version of the NDK as you used, they are compatible. If it was not, it depends. To my knowledge the only major incompatibility is for any pre-r11 NDK and post-r11 NDK where there was an ABI break. Other versions might be feature (if oboe is using libc++ APIs that were not available in the version used by libpdfium, you need the newer version) or bug (same thing, but behavioral differences rather than missing APIs) incompatible. We don't know of any such issues but that doesn't mean they don't exist. Unless you have reason to do otherwise, prefer whichever one comes from a newer NDK.
This is a good reason to package libraries as AARs, btw. The tool that manages those has that sort of compatibility knowledge baked in so you don't need to worry about it. You'll get the version from your project unless there's a conflict, in which case you'll get an error message if the two are actually mutually incompatible (at which point you either need to raise a bug with your dependencies to get them compatible or drop a dependency).
Why isn't it documented?
Good question! Filed http://b/182306131 (sorry, I couldn't find the publicly visible component, but it just points back to this thread so you're not missing anything).
@DanAlbert - Thanks for responding here. You wrote
the dependencies could have been built differently
Should we be packaging the Oboe PreBuilt binary differently? Should we leave out libc++_shared?
Oboe is actually the only dependency here that's already doing the right thing :) It's the other two that are causing the problem.
Wow thanks for the detailed reply.
Currently my android library (the one I showed you the build.gradle for) is distributed as an .aar. I checked this by looking in my maven repo; also the fact that it's built using apply plugin: 'com.android.library'
I checked the pdfium dependencies:
- android-pdf-viewer (the thing my app depends on) is distributed as an .aar also (uses AGP 3.4.2)
- pdfium (the thing android-pdf-viewer depends on) is distributed as an .aar also (uses AGP 3.1.3)
So it looks like everything is using AAR. Could it be that the version of Android Gradle Plugin used to build those libraries was too old? If I could get those libraries to be built with a more recent version of AGP should it just work?
Side-question: it seems like I have the option in my library to use oboe with a statically linked version of C++, i.e. arguments "-DANDROID_STL=c++_static" however this doesn't build (Cause: executing external native build for cmake). I'm guessing that's because Oboe requires c++_shared, but I don't understand why some samples are able to use c++_static?
I'll let @DanAlbert answer the main question but I'll try to answer your side question:
The samples are able to use c++_static because they're building Oboe from source, rather than using the Oboe binary.
You cannot use that option in your library because the Oboe binary is dependent on c++_shared and you cannot link to both c++_shared and c++_static.
Currently my android library (the one I showed you the build.gradle for) is distributed as an .aar. I checked this by looking in my maven repo; also the fact that it's built using apply plugin: 'com.android.library'
Whoops, yes, I used the wrong terminology here while trying to avoid getting into implementation details (and misunderstanding this to be a C++ API).
This is an interesting case. To clarify: libnativeMidiSynth is a Java API that uses oboe for its implementation? Or is libnativeMidiSynth a C++ API? My original response assumed that latter.
If it is in fact a Java library, https://android.googlesource.com/platform/ndk/+/master/docs/user/middleware_vendors.md#for-java-middleware-with-jni-libraries applies, but you can't follow that advice unless Oboe provides a static library. That's theoretically quite simple.
@dturner if you go this route (seems like a good idea to me), just add additional modules to the prefab package for the static variants, e.g. oboe and oboe_static. Here are a couple examples:
- https://github.com/google/prefab/tree/master/cli/src/test/resources/com/google/prefab/cli/packages/static_and_shared
- https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-master-dev:build-system/integration-test/test-projects/prefabPublishing/foo/build.gradle
You can use export_libraries to declare the dependencies on liblog and libOpenSLES for the static variants (unnecessary for the shared variants because they're already linked, but doing this for the static variants means your users don't need to worry about linking your dependencies).
So it looks like everything is using AAR.
Ah, sorry, I was basing that advice on https://pdfium.googlesource.com/pdfium/, which looks to be C++ only. Didn't realize this was a Java wrapper around it. In that case https://android.googlesource.com/platform/ndk/+/master/docs/user/middleware_vendors.md#for-java-middleware-with-jni-libraries applies.
Manual conflict resolution in your build.gradle like you currently have is unfortunately probably going to be common since so many libraries don't follow that advice. I wonder how feasible it would be to add a warning/lint check to AGP for packages that don't follow that advice :thinking:
Ah yes, sorry, I think all the libraries are Java wrappers around C++ libraries. I guess I don't know how to do different, but in the case of libnativeMidiSynth, that's just one piece of that android library (there's also a sequencer and whatnot written in Java). I'm not exposing the C++ as part of the library's interface - just the Java.
I suppose since I have this workaround this isn't so urgent, but I will try to digest those links over the next few days. Thanks for the help!
To be clear I don't think there's anything you can currently do on your end (unless you're the one that built those pdfium libraries as well). Probably worth a read anyway, but it sounds like your current configuration is correct.