oboe
oboe copied to clipboard
google
Significant number of crash reports from Oboe library
Android version(s): 9 and 10 but 90% from Android 10 Android device(s): Assortment of devices Oboe version: 1.5.0 App name used for testing: M64Plus FZ
Short description
Library causes a crash when calling requestStart or start
Steps to reproduce N/A
Expected behavior It doesn't crash
Actual behavior Random crashing
Device Assortment of devices
Any additional context I'm getting various core dumps through crashlytics, here are a couple:
Crashed: Thread : SIGSEGV 0x000067fd3612f1c0
#00 pc 0x75f01a5774 libaaudio.so
#01 pc 0x757b09bc30 liboboe.so (oboe::AudioStreamAAudio::requestStart() [AudioStreamAAudio.cpp:382])
#02 pc 0x757b09bc30 liboboe.so (oboe::AudioStreamAAudio::requestStart() [AudioStreamAAudio.cpp:382])
#03 pc 0x757b09d05c liboboe.so (oboe::AudioStream::start(long) [AudioStream.cpp:107])
#04 pc 0x757b0e8098 libmupen64plus-audio-android.so (AudioHandler::resumePlayback() [AudioHandler.cpp:558])
#05 pc 0x757f2d1e70 libjnidispatch.so
Crashed: Thread : SIGSEGV 0x0000000000110074
#00 pc 0xdf05a672 libaaudio.so
#01 pc 0xbb4dc41f liboboe.so (oboe::AudioStreamAAudio::requestStart() [AudioStreamAAudio.cpp:382])
#02 pc 0xbb4dd30b liboboe.so (oboe::AudioStream::start(long long) [AudioStream.cpp:107])
#03 pc 0xbb5638ed libmupen64plus-audio-android.so (AudioHandler::resumePlayback() [AudioHandler.cpp:558])
#04 pc 0xbebe6ce2 libjnidispatch.so
#05 pc 0xbb5628ff libmupen64plus-audio-android.so (pauseEmulator [plugin.cpp:494])
Only the top most part of the back trace is included.
This is a high priority bug. Please help us by providing more information.
What are some of the devices that are crashing? What are some of the devices that are NOT crashing? Have you ever seen this crash in your office?
The stack dump mentions AudioHandler::resumePlayback(). Is that only called when restarting a stream or it also called when stream is first started?
How are you currently handling onError calls when headsets are unplugged?
Are you using an std::shared_ptr when you call openStream()? That is important.
Does it always crash at [AudioStreamAAudio.cpp:382])? or other places as well?
Please share any more information you can get from the crash reports.
I ran M64Plus and can see that you are getting an exclusive MMAP stream on Q on Pixel 1.
Here are the devices that are crashing:
- Galaxy Tab A (8.0", 2019)
- Chromecast Google TV
- Redmi 7 -- Android 9/10
- Galaxy Tab A
All are running Android 10 except for Redmi 7 where I had one crash report with Android 9.
Those are the ones that I have reports on so far. Keep in mind, it's 8 crash reports so far out of thousands of users.
It would not be worthwhile to list devices that are not crashing since I have a large user base.
I have never personally seen this crash.
resumePlayback is called when the app is paused, then resumed, not on first start.
I'm not currently handling error calls when headsets are unplugged. I have not personally tested that, but it sounds like it should be something that I'm testing.
I'm currently using this type for my stream: ManagedStream
Looking through the Oboe code, it appears that it's a typedef to std::unique_ptr<AudioStream, StreamDeleterFunctor>
I'm using oboe::AudioStreamBuilder to open the stream. See code here: https://github.com/mupen64plus-ae/mupen64plus-ae/blob/master/mupen64plus-audio-android/src/AudioHandler.cpp
All the crash reports so far have [AudioStreamAAudio.cpp:382]).
I'm currently using this type for my stream: ManagedStream
We have found potential race conditions associated with using ManagedStream or using the raw stream pointer. Those are now deprecated. It could cause these kinds of crashes.
Please use std::shared_ptr. I think that will fix it. If not then please open a new bug.
(I just noticed the GettingStarted guide still references ManagedStream!)
By the way, you have this commented out:
//builder.setSampleRateConversionQuality(oboe::SampleRateConversionQuality::Fastest);
You can get significantly lower latency for other sample rates if you enable it and use Medium quality:
builder.setSampleRateConversionQuality(oboe::SampleRateConversionQuality::**Medium**);
Yep, I'm aware. The reason I have it commented out is because it causes performance penalties on very low end devices.
The built in resampler has less overhead.
Thanks for the note on performance. On Pixel I found that it improved performance.
I updated the GettingStarted guide: https://github.com/google/oboe/blob/master/docs/GettingStarted.md#creating-an-audio-stream
@philburk I have updated my audio implementation to use the shared pointer as suggested. I'm still getting a very significant number of crash reports coming from the oboe library.
You can see the updated implementation in the link I provided previously.
If you would like, I could provide you with selective access to my firebase project so that you can see the crashlytics reports. I haven't tried that yet, but I believe firebase lets you do that, so I can pursue that if you would like.
@fzurita - Yes, I would be interested in seeing that Firebase information.
Are you able to reproduce the crash?
Are you deleting the shared stream? What if you keep the old one until you open a new one?
@philburk I would like to give you access to my firebase project, just the crashlytics portion. To do that I need an e-mail from you, if you can provide me an e-mail, I will give you access. If you wish to e-mail me your info, you can find that in my github profile.
To answer your questions, I'm not able to reproduce these crashes, they appear random, maybe race conditions. I'm not deleting the shared stream, I do reopen the stream using openStream if initialization fails.
@philburk were you able to view the craslytics page? I never got confirmation.
No, sorry. I got swamped again this week. (This is why we are trying to hire a new engineer.) I will try next week.
I'm having similar issue when calling requestStart(). Phone is a Redmi 9 with Android 10.
I'm using react-native and it does crash too when launching the app in a simulator using Pixel_3a_API_30_x86.
The app crashes with SIGSEGV and SIGSERR errors. Sometime I get SIGBUS and BUS_ADRALN errors.
At first I thought I was misusing the void pointer argument audioData of the onAudioReady method, so I just took the codes in the get started guide and it produced the same error.
Lastly, I just called the method without anything inside but returning DataCallbackResult::Continue and it still reproduced the same error.
What I'm sure of is that the method is called once and there is a brief sound played corresponding to the data I feed the audioData pointer. There is no problem when writing to the pointer and what I write correspond to what I hear briefly.
Oboe is version 1.5.0 and I tried using version 1.4.3 but with the same result.
Hope this get resolved soon and thank you for looking at it.
I was still debugging my app and I finally found the bug was due to my own error. I declared the Object containing the audio stream variable inside a function that terminated once the callback has terminated, and thus it was deleted before the next call of the onAudioReady callback. I just move that declaration to the global scope to see any changes and the problem disappeared. I apologize if my comment caused any problem to the resolution of this issue.
Thanks for the update. We should warn against declaring the stream variable in a method. I added that to:
https://github.com/google/oboe/blob/master/docs/GettingStarted.md#creating-an-audio-stream
Maybe this one shouldn't have closed since he was replying to my old issue.
The app has 500,000+ downloads.
I am looking at the Firebase reports. In Oboe 1.5.0, AudioStreamAAudio.cpp line 382 is:
return static_cast<Result>(mLibLoader->stream_requestStart(stream));
From the stack trace, it seems to be crashing in libaaudio.so, probably under AAudioStream_requestStart(), but we do not have any stack trace from libaaudio.
Almost all of the Samsung crashes are on the Galaxy Tab A. It does not support MMAP so it is not MMAP related.
@fzurita - Is your app much more likely to be run on a tablet than on a phone? I see a lot of TV and tablets devices crashing. There are no crashes on Pixel phones.
I notice that AudioStreamAAudio::requestStart() is locking on mLock but not locking on mAAudioStreamLock. But the comments in AudioStreamAAudio::close() suggests it should be locked. But given that we lock on mLock. I do not see how requestStart() could use a stream that had been closed.
I am playing a game and noticed that the stream seems to be started twice:
05-25 18:04:35.886 17560 17615 D AAudio : AAudioStream_requestStart(s#2) called -------------- 05-25 18:04:35.887 17560 17615 D AAudio : AAudioStream_requestStart(s#2) returned 0 --------- 05-25 18:04:35.888 1010 1010 D AudioPlayerStateMonitor: Found a new active media playback. ID:199 -- type:AAudio -- u/pid:10131/17560 -- state:started -- attr:AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_UNKNOWN flags=0x0 tags= bundle=null 05-25 18:04:35.936 17560 17560 D AAudio : AAudioStream_requestStart(s#2) called -------------- 05-25 18:04:35.936 17560 17560 D AAudio : AAudioStream_requestStart(s#2) returned -895 ---------
I tried plugging and unplugging headphones while the game was running. I could not cause a crash.
The Galaxy Tab A is running QP1A.190711.020
My app is mostly ran on phones but there are many users in TVs, tablets, and even chromebooks.
I may have to look at that issue with starting the stream twice. I'm not sure how that could be happening, but I can investigate.
I do try to restart the stream in case of failures since there are some devices that sometimes fail to start the stream but succeed on retry (I think this was on chromebooks).
For reference, you can find the source code here:
https://github.com/mupen64plus-ae/mupen64plus-ae/blob/master/mupen64plus-audio-android/src/AudioHandler.cpp
If you want to see the details on how I use the library.
This is a curious one too, oboe appears to sometimes cause a deadlock, which causes an ANR:
https://console.firebase.google.com/project/mupen64plus-fz-pro/crashlytics/app/android:org.mupen64plusae.v3.fzurita/issues/628ebe81f95c12b96a003558f32839f2?time=last-seven-days&versions=3.0.281%20(beta)-free%20(281)&sessionEventKey=60ac56ae032900012225f980f74b8d16_1544482718339006776
I seem to have a lot of these too (ANRs):
#00 pc 000000000004bd0c /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
#00 pc 000000000004fb68 /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+144)
#00 pc 00000000000b7068 /apex/com.android.runtime/lib64/bionic/libc.so (NonPI::MutexLockWithTimeout(pthread_mutex_internal_t*, bool, timespec const*)+232)
#00 pc 00000000000bf148 /data/app/~~lUtOMyiIKiTiwQxZ8Qwrtw==/org.mupen64plusae.v3.fzurita-jE1EMdj7t8TLtVGilTFf7Q==/lib/arm64/libc++_shared.so (std::__ndk1::mutex::lock()+8)
#00 pc 000000000001bbd0 /data/app/~~lUtOMyiIKiTiwQxZ8Qwrtw==/org.mupen64plusae.v3.fzurita-jE1EMdj7t8TLtVGilTFf7Q==/lib/arm64/liboboe.so (oboe::AudioStreamAAudio::requestStart()+28)
#00 pc 000000000001d05c /data/app/~~lUtOMyiIKiTiwQxZ8Qwrtw==/org.mupen64plusae.v3.fzurita-jE1EMdj7t8TLtVGilTFf7Q==/lib/arm64/liboboe.so (oboe::AudioStream::start(long)+28)
#00 pc 000000000000c314 /data/app/~~lUtOMyiIKiTiwQxZ8Qwrtw==/org.mupen64plusae.v3.fzurita-jE1EMdj7t8TLtVGilTFf7Q==/lib/arm64/libmupen64plus-audio-android.so (AudioHandler::resumePlayback()+52)
For the ANR, we have seen ANRs in the HAL that can propagate up to the service and then the app.
Also I see you are calling AudioStream::start() with a timeout. You may want to call the asynchronous restartStart(), requestStop() and related methods instead. I have seen ANRs in code that waits for the state change. Eg. #1065
This Firebase info is very helpful as far as telling us the frequency, OS, device and other stats for crashes. But the lack of detailed stack traces, system logs ands tombstones make it hard to debug.
Yeah, unfortunately there is no easy way to plug logcat output to firebase logging.
I replaced start with requestStart but ANRs have not been resolved. I guess I could move the requestStart into a separate thread, Here it is for reference and you can use that to traverse my source code in the provided link if interested:
#00 pc 000000000008033c /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
#00 pc 0000000000083990 /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+140)
#00 pc 00000000000e76cc /apex/com.android.runtime/lib64/bionic/libc.so (NonPI::MutexLockWithTimeout(pthread_mutex_internal_t*, bool, timespec const*)+228)
#00 pc 00000000000bf148 /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libc++_shared.so (std::__ndk1::mutex::lock()+8)
#00 pc 000000000001bbd0 /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/liboboe.so (oboe::AudioStreamAAudio::requestStart()+28)
#00 pc 000000000000c30c /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libmupen64plus-audio-android.so (AudioHandler::resumePlayback()+44)
#00 pc 0000000000010154 /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libjnidispatch.so (???)
#00 pc 000000000000f938 /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libjnidispatch.so (???)
#00 pc 0000000000005cfc /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libjnidispatch.so (???)
#00 pc 0000000000007bc8 /data/app/org.mupen64plusae.v3.fzurita-afM19L_zzvnY4_oi4xK4iw==/lib/arm64/libjnidispatch.so (Java_com_sun_jna_Native_invokeVoid+32)
at com.sun.jna.Native.invokeVoid (Native.java)
at com.sun.jna.Function.invoke (Function.java:415)
at com.sun.jna.Function.invoke (Function.java:361)
at com.sun.jna.Library$Handler.invoke (Library.java:265)
at java.lang.reflect.Proxy.invoke (Proxy.java:1006)
at paulscode.android.mupen64plusae.jni.AndroidAudioLibrary.resumeEmulator (AndroidAudioLibrary.java)
at paulscode.android.mupen64plusae.jni.CoreInterface.emuResume (CoreInterface.java:699)
at paulscode.android.mupen64plusae.jni.CoreService.resumeEmulator (CoreService.java:263)
at paulscode.android.mupen64plusae.jni.CoreFragment.resumeEmulator (CoreFragment.java:395)
at paulscode.android.mupen64plusae.game.GameActivity$1.onDrawerClosed (GameActivity.java:421)
at androidx.drawerlayout.widget.DrawerLayout.dispatchOnDrawerClosed (DrawerLayout.java:859)
at androidx.drawerlayout.widget.DrawerLayout.updateDrawerState (DrawerLayout.java:829)
at androidx.drawerlayout.widget.DrawerLayout$ViewDragCallback.onViewDragStateChanged (DrawerLayout.java:2174)
at androidx.customview.widget.ViewDragHelper.setDragState (ViewDragHelper.java:891)
at androidx.customview.widget.ViewDragHelper$2.run (ViewDragHelper.java:343)
at android.os.Handler.handleCallback (Handler.java:883)
at android.os.Handler.dispatchMessage (Handler.java:100)
at android.os.Looper.loop (Looper.java:237)
at android.app.ActivityThread.main (ActivityThread.java:8107)
at java.lang.reflect.Method.invoke (Native method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:496)
at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1100)
Actually, this probably has to do with requestStart being called twice. I must be calling the second requestStart before the first one is complete.
I'm having similar issue when calling
requestStart(). Phone is a Redmi 9 with Android 10. I'm using react-native and it does crash too when launching the app in a simulator using Pixel_3a_API_30_x86. The app crashes with SIGSEGV and SIGSERR errors. Sometime I get SIGBUS and BUS_ADRALN errors. At first I thought I was misusing the void pointer argumentaudioDataof theonAudioReadymethod, so I just took the codes in the get started guide and it produced the same error. Lastly, I just called the method without anything inside but returningDataCallbackResult::Continueand it still reproduced the same error. What I'm sure of is that the method is called once and there is a brief sound played corresponding to the data I feed theaudioDatapointer. There is no problem when writing to the pointer and what I write correspond to what I hear briefly. Oboe is version 1.5.0 and I tried using version 1.4.3 but with the same result. Hope this get resolved soon and thank you for looking at it.
@Animasolus - as a minor aside, and had trouble contacting you directly; as there's very little obvious on the web, if you could point to any direction on how to go about using Oboe and React Native successfully it would be greatly appreciated!
@fzurita - There has been no discussion on the original issue for a while. Closing as obsolete. If you are still seeing crashes please open a new bug.
I'm having similar issue when calling
requestStart(). Phone is a Redmi 9 with Android 10. I'm using react-native and it does crash too when launching the app in a simulator using Pixel_3a_API_30_x86. The app crashes with SIGSEGV and SIGSERR errors. Sometime I get SIGBUS and BUS_ADRALN errors. At first I thought I was misusing the void pointer argumentaudioDataof theonAudioReadymethod, so I just took the codes in the get started guide and it produced the same error. Lastly, I just called the method without anything inside but returningDataCallbackResult::Continueand it still reproduced the same error. What I'm sure of is that the method is called once and there is a brief sound played corresponding to the data I feed theaudioDatapointer. There is no problem when writing to the pointer and what I write correspond to what I hear briefly. Oboe is version 1.5.0 and I tried using version 1.4.3 but with the same result. Hope this get resolved soon and thank you for looking at it.@Animasolus - as a minor aside, and had trouble contacting you directly; as there's very little obvious on the web, if you could point to any direction on how to go about using Oboe and React Native successfully it would be greatly appreciated!
@semsion that goes way back. The short answer is: I was using JNI as a way of communicating between C++ and Java, then React Native's native modules (Android) as a way of communicating between Java and React Native. A long and a little bit complicated answer, if you want to transfer the audio data from Oboe to React Native then:
- C++ -> Java: I stored it in an array of
charand then casted it tojbyteArray.
string audio; // where the audio is stored using oboe
extern "C"
JNIEXPORT jbyteArray JNICALL
Java_com_someproject_AudioStuffNative_getAudio(JNIEnv *env, jobject thiz) {
const char *audio = audio.c_str();
const int ARR_SIZE = (int)audio.size();
jbyteArray arr = env->NewByteArray(ARR_SIZE);
env->SetByteArrayRegion(arr, 0, ARR_SIZE, reinterpret_cast<const jbyte *>(audio));
return arr;
}
then in the Java module:
public class AudioStuffNative {
static {
System.loadLibrary("audio-module");
}
public byte[] native getAudio();
}
- Java to React Native, I encoded it to base64 to avoid data loss:
public class AudioStuffModule extends ReactContextBaseJavaModule {
AudioStuffNative audioStuffNative;
AudioStuffModule(ReactApplicationContext context) {
super(context);
this.audioStuffNative = new AudioStuffNative();
}
@ReactMethod
public void getAudio(Promise promise) {
try {
String base64Content = Base64.encodeToString(
this.audioStuffNative.getAudio(),
Base64.NO_WRAP
);
promise.resolve(base64Content);
} catch (Exception ex) {
ex.printStackTrace();
promise.reject("Error", "Some error message");
}
}
}
For using C++ with Android SDK: Add C and C++ to your project For the JNI part, I really don't remember. For the React Native Modules: React Native Modules. btw, I know almost nothing about Java programming.
This is great discussion about Oboe and React Native. But unfortunately it is buried in a bug report about crashes. So no one will find it. Please move this info to #955