App is using hardcoded 2 channels for input which will result in increased latency

[dturner]

There is a bug in Android O (internal id 66967812) which means that requesting a stereo input stream may result in a non-FAST track, resulting in increased latency on Android O devices.

Checking the logcat output on an Android O device should verify this.

Solution/workaround: Request only a single (mono) channel input stream.

Thanks to philburk@ for pointing this out.

[superpoweredSDK]

Thank you for the info. Is there any news about the grouped input/output callbacks?

[dturner]

I just fixed this issue in the AAudio echo sample here: https://github.com/googlesamples/android-audio-high-performance/commit/d269d62921d41f553cc742e4ea21b9d7c1789466

This is almost certainly the cause of the high round-trip latency being reported by this app on Android O.

Not sure what you mean by grouped input/output callbacks? Are you referring to synchronous I/O? If so, this is not supported yet. Best practice is to perform input stream reads during the output stream callback (see the echo sample for code).

[superpoweredSDK]

No, the grouped input/output callbacks are a different thing. I already explained this multiple times, over Twitter, in email and also here in GitHub. But let's do this again:

Previously, input and output callbacks monotonically followed each other: input, output, input, output, ...

Currently, input and output callbacks are coming in this order: input, input, input, input, output, output, output, output, input, input, input, input, ...

The size of the input/output groups changes between 3 and 5.

[dturner]

Sorry I have no idea why that behaviour has changed, you are welcome to look at the Android source to investigate: http://androidxref.com/.

My advice is rather than having an input stream callback, perform a non-blocking read from the output stream callback, then you won't have this issue.

[superpoweredSDK]

Yes, we are doing non-blocking read just like you describe, with AAudio. But I'm talking about OpenSL ES here. It's latency is significantly higher, negatively impacting all pro audio apps.

[superpoweredSDK]

Using mono input for AAudio reduced the round-trip audio latency to 38 ms (built-in mic/speaker) and 21 ms (loopback) on the Nexus 6P. These values are just slightly higher than OpenSL ES on Android 7. Meaning that AAudio provides no latency benefit yet, on this device.

[dturner]

Please raise a separate issue for the "i/o callback grouping", talking about two different issues on the same thread is confusing.

For the 21ms value - is that lower than it was without using the mono input?

[superpoweredSDK]

Yes, much better, 57/40 became 38/21 (built-in/loopback). Still not 33/17 though.

I'm not sure opening an issue here. Is this the appropriate place for it? The issue is not in the latency measurement app, but in Android.

[dturner]

OK I strongly suggest the following course of action if you want this fixed:

• Create a simple test app with source code available on github which reproduces the issue
• File an issue with clear steps to reproduce the issue at https://issuetracker.google.com/issues?q=componentid:192705%2B
• Send a link to the issue to me and I will raise internally.

FYI it's only those apps which rely on real-time audio monitoring which would be affected by this issue. That's still a big deal, just saying it's not all pro audio apps.

[superpoweredSDK]

Hi Don,

As you state, "it's a big deal" and big deal is within Android, not Superpowered.

As such, this is clearly an issue that the Android audio team should want to fix -- since you're on the team, and know about the issue, by all means.

Given how our earlier issues were handled, we're not very motivated to create an issue and also a test application. We already spent a lot of time and energy with this.

We've also sent our findings on multiple channels, via GitHub, email and Twitter DM. Unfortunately, we're not able to do significantly more at this moment -- we are small, Google is big.

[dturner]

"...and know about the issue" - The reason for asking you to submit that information in a bug report is because I don't know enough about the issue.

Here's some immediate questions I have:

• What is the impact of having 4 input buffer callbacks followed by 4 output buffer callbacks? Presumably increased latency. If so, by how much?
• Is there a way of avoiding this problem by using non-blocking reads inside the output buffer callback, rather than having 2 separate callbacks?

[superpoweredSDK]

I already mentioned a couple of times that non-blocking reads inside the output buffer callback is not possible with OpenSL ES. It's only possible with AAudio, and we are doing exactly that.

Please find all of our findings here: http://superpowered.com/android-audio-latency-problem-just-got-worse

[superpoweredSDK]

We provided tons of information and analysis via several channels in the past few weeks. We hope that turning them to the public will help your team getting the resources and attention for a proper fix, just like it happened in the past with the "10 ms problem" article.

[dturner]

I shouldn't have closed this. From the article I read "So we fixed our latency measurement app and everybody was curious..." and assumed that you'd fixed the input stream from stereo to mono, but it hasn't been fixed.

[dturner]

I have provided a fix/workaround to this issue here: https://github.com/superpoweredSDK/SuperpoweredLatency/pull/6

I should have picked up on this sooner but you thought this bug only applied to AAudio, when in fact it applies to OpenSL ES as well.

Now for some explanations:

1. The app was requesting a stereo input stream which because of bug 66967812 results in obtaining a non-FAST path
2. This non-FAST stream has burst sizes which are significantly larger than a FAST stream
3. The app assumes it has a FAST stream and in each callback write a single burst of data which contains only FRAMES_PER_BUFFER audio frames
4. The non-FAST stream still has more data to write so makes immediate successive callbacks until its buffer is empty
5. Meanwhile the output stream is FAST and is ticking along making regular callbacks every ~4ms
6. The different callback periods of input and output streams causes the irregular pattern of callbacks IN-IN-IN-IN-OUT-OUT-OUT-IN-IN etc

[superpoweredSDK]

Thank you for the explanation, it's helpful and makes sense.

However, we think that switching to mono just to get nicer numbers is a cosmetic patch. A white lie.

Since there is no way for user apps to detect if they need to switch to mono input for the lowest latency or not.

There are also apps that can not switch to mono input at all. Think about a DJ app with DVS input. A single DVS signal requires 2 channels. Or multi-channel recording with live feedback.

[dturner]

If you have an audio device with mono input and a stereo output it makes sense to create streams with those properties.

For determining what audio devices are attached and their properties you can use the AudioManager.getDevices() and AudioDeviceInfo APIs

[superpoweredSDK]

1. That API is available from 23 only.
2. But the biggest problem is: it returns 1, 2, and 3 channel options on the Nexus 6P for audio input TYPE_BUILTIN_MIC.

[dturner]

For now the suggested workaround is to only request a mono input on Android 8.0. I have updated my pull request to this effect: https://github.com/superpoweredSDK/SuperpoweredLatency/pull/6/commits/c6a7471dc7e97771b163795a7a2b0695b024d677

Tested on Pixel running Android 7.1 and Nexus 6P running Android 8.0

We'll need to see whether the OEM releases of Android 8.0 include the AOSP fix before deciding if a more complex workaround is required (e.g. only enabling this workaround if the version of Android doesn't include the AOSP fix).

[superpoweredSDK]

We hope it's not too late for most of them. Fingers crossed!

[dturner]

Been giving this some thought. Respecting the fact that you want to provide a completely accurate picture of latency to developers and users, perhaps a good solution would be to add the ability to select the channel count for both input and output streams. There is at least one other scenario where latency might be different between stereo and mono:

When AAudio sharing mode is set to EXCLUSIVE, it will (for now) only be granted if the output stream is in stereo.

I am also removing my comment about the article being "misleading" - it was said in the heat of the moment and wasn't constructive, my apologies.

[mail2chromium]

Can you please assist me with this issue?

https://github.com/superpoweredSDK/SuperpoweredLatency/issues/11#issue-925958195