Audio Latency Still An Issue In 2022 [Android]

[dmunsie]

I'm shocked this is still even an issue. It's not an "Android" problem because I've made test apps with other languages that do not have any audio lag when playing audio. Can I please get an official response if this is on the radar to get fixed? Thank you.

[ajwfrost]

Hi

This may be due to how the runtime is doing the audio; internally it's often using some very low level mechanisms that give it a lot more capabilities, but at the cost of additional complexity that can introduce latency.

Can I check what you're actually trying to do, and what effect you're then seeing?

We have been looking at simplifying/updating the multimedia capabilities to provide a simpler way for people to access the "common cases" i.e. just "play this file" without needing to have the ability to dynamically receive and manipulate the audio samples or applying bespoke filtering etc.. so that should help bring it more to parity with other languages/frameworks - i.e. we'd just immediately call down to the high-level platform APIs to perform the media actions, rather than doing all the demuxing, decoding, filtering, mixing, etc etc, internally and then pushing the sound buffers out through the low level APIs.

thanks

[dmunsie]

Thank you for taking the time to reply. I greatly appreciate it.

"Can I check what you're actually trying to do....."

The most basic part of the sound class, simply playing a sound effect: s_soundchannel = s_sound.play(s_starttime, s_loops, s_soundtransform);

".....and what effect you're then seeing?"

There is a noticeable delay before the sound actually starts playing. Visually it's more obvious in games where you have bullets, missiles being fired from guns and the sound of the gun firing is delayed causing a loss of player immersion at least on the audio side. Or in a gui where the user would click on a button but the sound effect would not play right away, etc, etc.

Digital Strawberry developed an ane that helped with this delay a while back, it did work on older android devices, but they stopped support of the ane and now it has issues with modern hardware. The audio plays fine but it causes small jitters in program execution (verified with the latest runtime) making the ane useless as a stable solution. Perhaps you can download their ane to give you some ideas how to correct the issue.

https://github.com/DigitalStrawberry/ANE-Sounds

Here are discussions about the audio lag. Every perceived "solution" simply doesn't work across the board so a stable solution provided by the runtime would be a welcome addition.

https://community.adobe.com/t5/air-discussions/how-to-fix-sound-delay-in-adobe-air-apps/m-p/6437649 https://community.adobe.com/t5/air-discussions/how-to-fix-sound-delay-in-adobe-air-apps/td-p/6437643 https://community.adobe.com/t5/air-discussions/best-settings-for-audio-play-latency/td-p/3415506 https://stackoverflow.com/questions/5690118/sound-latency-in-as3

[ajwfrost]

Thanks - have just had a quick look at the ANE there, it uses the Android "SoundPool". There may be some challenges in how it's used with file paths etc with the latest Android versions, but any glitches/jitters would likely be due to the threading (i.e. they're still running things in the main UI thread and decoding MP3s etc could cause delays to the AIR rendering that's also happening in that thread). https://developer.android.com/reference/android/media/SoundPool

A lot of other issues mentioned there - but the main use case you're talking about is going to be one that is quite important to a lot of people! so we'll see what we can do about that, there's no real reason for this case to be slow...

[dmunsie]

"A lot of other issues mentioned there - but the main use case you're talking about is going to be one that is quite important to a lot of people! so we'll see what we can do about that, there's no real reason for this case to be slow..."

Awesome, thank you! :)

[dmunsie]

Is it possible to get an update / eta on resolving this issue? It's long.....long.....over due and I guarantee will make a lot of game developers very happy when this is resolved. Thanks!

[pnico]

for low-latency audio in Android there's this https://developer.android.com/ndk/guides/audio/aaudio/aaudio but it's only in android v27+ so they recommend this for android 16+ compatibility: https://github.com/google/oboe there's no audio decompression, only plays PCM audio, but I assume AIR has decades of code for that already in it?

[teotigraphix]

Hey @ajwfrost this just was a question, since I don't know how the runtime hooks into the audio system but, how hard would it be to expose some type of C++ callback in the FREContext for an ANE to implement an audio callback(in C/C++)?

Do you see where I am going with this? Is there any centralized place you are feeding the DSP from the AIR runtime to the bare metal audio callbacks? If this was done correctly or "in the correct place" you would get realtime audio and could feed information up to the AIR client the normal ANE way, from a "custom" audio engine(I have one as an example).

Something like this could really open up AIR, low latency audio callbacks are gold. You already have to funnel those audio samples into each platform's audio calbbacks somewhere... Anyway, this could be completely stupid as well, since I don't know what is going on in the black box of AIR.

[ajwfrost]

@teotigraphix am not completely sure I understand what you're suggesting, but do you mean that we could send decoded/raw audio data from the mixer within AIR out to an ANE? That could be a possibility .. internally, there are buffers of encoded data queued up and decoded (from each SoundChannel object) and then mixed into a single output (via SoundMixer) and these buffers are then passed to the platform-specific implementation.

We've been working on a new multimedia framework where it would be possible for ANEs to implement an element (or multiple elements) of the pipeline which would mean you could set up something to have audio sent to the ANE for output (pre-mixer if required, or whatever). Meant to be very flexible and powerful but is proving fairly tricky to implement on a cross-platform basis hence taking a while - so we were looking to also short-cut this to provide simpler APIs for the very common use cases...

[teotigraphix]

@ajwfrost I don't know why I am not getting notifications but, your second paragraph is exactly what I was trying to say.

Have an API that an ANE can implement pre/post audio buffer callbacks, you know that is putting it simple. :) This way, my audio engine, as long as the sample rate jives, could pump it's whole audio stream into your buffers on down at the raw metal level. This would be insane if you could get it to work, it would make audio a first class citizen with AIR.

I have so much boiler plate just getting to a buffer, and multi-platform buffer callbacks as you can see are FUN!

[dmunsie]

Imagine shooting missiles, collecting coins, explosions, etc, during your game but the audio is always a bit behind the action due to the audio lag that happens when launching a sound via the normal soundchannels available in the audio system.

It's now 2024 and this long time bug on Android via the Air/SDK is still very much alive and well. Every time I mention this it seems like only a a couple of people respond, so either the reality is no one is making games using Air/SDK on Android or everyone is either accepting the lag or.... there's a solution I am not aware of. hmm.... Thank you for any comments, solutions, progress, etc.

[ajwfrost]

Hi

I think a lot of people are likely using ANEs/workarounds.. just re-reading the above thread, and I'd made the comments about the new multimedia APIs... which was 18 months ago. They are creeping forward slowly, but unfortunately we've been more bogged down in necessary maintenance activities to support the ever-changing requirements/updates from Apple/Google so the progress isn't as good as I would have hoped.

Some of the capabilities are in there, but only for specific platforms, so (and I'm aware I'm repeating myself) we could try to focus on that common case of a simple sound effect on Android, and at least get that working....

thanks

[dmunsie]

Thank you for commenting. "I think a lot of people are likely using ANEs/workarounds." I wish I knew what these possible solutions were because I have searched over and over and nothing comes up. As I mentioned earlier, the DigitalStrawberry ANE did improve things dramatically years ago so it is possible, but that ANE was never updated to work properly with newer SDKs.