Audio quality on iOS 6

[emiliopavia]

Hi, the audio quality get worse when using the library on iOS 6. I know that you disabled the echo cancellation, do you have any plan for reenabling it in a near future? Are there other known issues with iOS 6?

Thanks a lot!

[mkrautz]

The only issue I know of is that the VoiceProcessingIO audio unit produces zero amplitude samples on iOS 6 in the configuration that MumbleKit uses (and, I believe, all configurations). When and if I figure out why it's doing that, I'll re-enable for iOS 6.

That's the only issue I know about regarding iOS 6 compatibility.

Are you using MumbleKit in a shipping app? That'd be nice to know, since I'd be able to warn you about any nasty showstopper-ish bugs I run into in the future.

[mkrautz]

Also, do you believe the quality issue you're talking about is unrelated to the turning off of Echo Cancellation?

[emiliopavia]

Well, actually we had to ship today, but we found that the build that Apple approved us wasn't working on iOS 6 (audio in/out muted). After recompiling it with the new Xcode 4.5 the audio started working again.

As for the quality we noticed that using two devices, the one with iOS 5 has a good audio quality, while audio in the other one with iOS 6 was poor, but we are not sure if it depends on the OS or something else. We were using speex @ 32Kb/s.

[mkrautz]

I found this in my iPhone 4S console:

Sep 21 18:44:25 Mikkel-Krautzs-iPhone mediaserverd[6206] <Notice>: >vad< WARNING: 18:44:25.972 [tid 0x20ff000] VirtualAudio_PlugIn.cpp[134]: SetVoiceProcessingState(): Attempt to enable routing in support of voice processing while in or requesting an invalid route/mode pair ([ Category: 'cpar'; Mode: 'imdf' ]). Sep 21 18:44:26 Mikkel-Krautzs-iPhone mediaserverd[6206] <Error>: 18:44:26.158 <AudioControl> _vp: initialize hw input: ul-dl config still out of sync ?!

[mkrautz]

'cpar', I think, means Category Play and Record.

'imdf' I'm not sure of.

[emiliopavia]

I found something really strange. I tried to build our app using the MumbleKit sources at the commit bf23e9d250d476f53ce91bf7523000c6fd8daa35 (so before disabling the echo cancellation). If I build that using a release configuration (i.e. for an ad hoc distribution) audio doesn't work and there's only silence. If I build and run with Xcode (in debug mode) audio passes in both the devices. Any idea about that?

Edit: of course using Xcode 4.5 and iOS 6

[mkrautz]

So you're saying that EVERYTHING works as expected if you build in your app's Debug configuration?

[emiliopavia]

Actually I couldn't test the echo cancellation yet. I should do that laters but now I can hear in the simulator the voice coming from the device (I don't have two physical devices right now).

[mkrautz]

I'm confused. Does it have echo cancellation enabled? That is, are you saying that you simply can't do the two-device test, to test whether the echo cancellation is effective?

Because echo cancellation is definitely what breaks it for me.

[emiliopavia]

Yes, it has echo cancellation enabled, but I cannot test if it actually works as expected. What I cannot understand is why audio works in debug configuration and don't (aka silence) in release configuration.

Have you tried to rebuild the Mumble client (using the "old" sources) and running with Xcode 4.5?

[mkrautz]

Interesting. I believe I still had the bug when I built it using the old sources.

I'll try experimenting.

[mkrautz]

I can't seem to reproduce what you're seeing here, unfortunately. Are you sure your app has Echo Cancellation enabled? It needs to set MKAudioSettings.enableEchoCancellation to trigger MKAudioInput to use the VoiceProcessingIO audio unit.

[emiliopavia]

Yes, I have that enabled.

[mkrautz]

It might be Speex. For Speex, MKAudioInput uses a 32000Hz sampling rate.

[mkrautz]

(That was just me thinking aloud... It's still using 48KHz on the audio unit itself...)

[mkrautz]

I still can't seem to reproduce this. I've tried using bf23e9d as you say (with Xcode 4.5 GM - 4G182 - is that the build you're using as well? Anyway, I can't seem to update it via the Mac App Store either way.)

My Test projects schemes are set up to run my Test.app in its Debug configurartion. No luck.

[emiliopavia]

Can you confirm that audio in/out with the old version of Mumble in the App Store was mute using iOS 6?

[mkrautz]

Yes, I'm a quite sure. I have the binary still, so to double-check, I'll re-sign it an run it on iOS 6 as if it were from the App Store.

Another data point: https://sourceforge.net/tracker/?func=detail&aid=3568656&group_id=147372&atid=768005

That's from before I released the 1.1.1 bugfix.

I'll be offline for a couple of hours though, so I'll do the testing when I get back.

[mkrautz]

I just re-tried my old 1.1 app on iOS 6 - still 'mute'. (So I'm not going crazy -- good.)

There's a thread discussing the issue on the dev forums: https://devforums.apple.com/message/710796

A radar has been filed: 12352954 (it's not available on OpenRadar). I'd recommend duping it on Radar to show its popularity.

Are you still able to fix the problem by simply running in the Debug configuration? (And have you tried whether the echo cancellation is effective?). Very interested in hearing about that.

If you're still stuck with this issue there's another possible solution. In Desktop Mumble we use the Speex echo canceller. It should be possible to integrate a similar system into MumbleKit, but only using Mumble's own played back audio (since we can't get at a system-wide audio output stream).

[mkrautz]

I've also created a Radar with all the info I know:

12354101

Feel free to duplicate both of these two bug IDs on https://bugreport.apple.com: simply mention that you're duping them in the summary.

[mkrautz]

@emiliopavia, just a heads up. I'm working on getting the Speex AEC into MumbleKit.

[emiliopavia]

Thanks! I am out right now for vacations but I will start again watching on that in some days.

Il giorno 23/set/2012, alle ore 13:11, Mikkel Krautz [email protected] ha scritto:

@emiliopavia, just a heads up. I'm working on the Speex AEC into MumbleKit.

— Reply to this email directly or view it on GitHub.

[mkrautz]

@emiliopavia You might find my latest commit interesting. I've shuffled the device management around a bit, and can now use a single audio unit for both input and output.

In the end, I couldn't get the Speex echo canceller to produce a good result, but that's just me not using it correctly, most likely. Probably not using the correctly delay for the filter, or something along those lines...

So I've settled on this solution for now. I see you have some hacks in your fork that do a smiliar thing, but doing it via a single AudioUnit should produce more satisfactory results.

Feel free to give it a spin.

[emiliopavia]

Thanks a lot, we'll have a look at that!

[antoniotuzzi]

Hello @mkrautz I did the same shuffling (just to test) (creating a "MKAudioInputOutput" class) and I had the same results. Everything can work using just the VPIO, but the volume is extremely LOW (at least half of the past till the iOS6 coming)

if you have access to the Apple forum check this discussion: https://devforums.apple.com/thread/162542?tstart=0 at this point I explain why the Echo cancellation was never activated on IOS6 https://devforums.apple.com/message/734868#734868 for this I opened a bug in radar 12375108.

I think that this is a bug from Apple: I don't understand why someone can't disable VPIO_output to get just samples from the microphone.

For me it's ok to use the new mode (I talking about Apple) with just or the VPIO or the RemoteIO. But audio volumes should be consistent

[mkrautz]

@doepferiano Thanks, I do read the dev forums, and I have a dup'ed the 12352954 bug for this issue as well.

But I'm afraid I don't see the low volume in my tests with these changes -- I only got a low output volume if I use two AudioUnits (one VPIO for mic input, one RemoteIO for output). But if I actually use the output side of the VPIO, I seem to get a fine output volume.

At least that's my recollection from last night. :-) If you try the latest mumble-iphoneos (if that's easiest for you), you can toggle between VPIO and RemoteIO by toggling the Echo Cancellation switch in Preferences.

I can't test it right now, but if there's still low output volume, that'd be a showstopper for me as well...

I'll do a proper test when I get home.

[mkrautz]

Also, is it the output or the input that's low volume? Because for me, when using my VPIO-based MKAudioDevice I seem to recall getting a higher volume, rather than lower.

EDIT: that's INPUT volume.

[mkrautz]

I've just tried to test again, and I do get (from what I can perceive) the same volume for both MKVoiceProcessingAudioDevice (which uses a single VPIO audio unit for both input and output) and for MKiOSAudioDevice (which uses a single RemoteIO audio unit for both input and output).

[antoniotuzzi]

Hello @mkrautz today I've done the test using your last iteration. Very good and CLEAN idea to maintain 2 separated audiodevices for the iPhone (VPIO and RemoteIO). Anyway the volume is lower than in the past. If you use your headphone you won't notice it, but if you put the iPad for example on a table at one meter from you and do a test against the version with mixed audiodevices (input using VPIO and output using RemoteIO) (the one working on on iOS5) there is a sensible difference!

[ksam917]

Hi all, I am trying to capture the audio using the VPIO units on capture and playback side. What I am facing is when the playback is happening, that time the captured audio is having the zero valued samples. This is giving the problems in double talk conditions (i.e., when both farend and nearend are active). This one I am doing on iOS 6 using XCODE 4.5 version. How to over come the problem of zero valued samples in the captured audio.