MumbleSharp icon indicating copy to clipboard operation
MumbleSharp copied to clipboard

Massive delay and skipping audio (jitter?) when using Android/Xamarin

Open BenWoodford opened this issue 1 year ago • 0 comments

I've just started working with this (thank you by the way) and have incorporated trevorirwin's fork for Android/iOS support too

I'm using Murmur and a PC mumble client on the LAN for testing right now with mixed results... I'm tackling output first and getting some rather peculiar results. This is a background service (well, foreground service pretending to be a background service...), the mic and playback do work so that's not an issue it seems.

If I speak into my PC client, it can take upward of 2 seconds to output on the Android device (a Meta Quest 2)

I've abstracted out some of the code but here's the important bits:

            audioBuffer = new byte[MumbleSharp.Constants.DEFAULT_AUDIO_SAMPLE_RATE * (MumbleSharp.Constants.DEFAULT_AUDIO_SAMPLE_BITS / 8) * MumbleSharp.Constants.DEFAULT_AUDIO_SAMPLE_CHANNELS / 2];
        public override void SetProvider(IWaveProvider provider)
        {
            base.SetProvider(provider);

            this.provider = provider;

            AudioTrack.Builder builder = new AudioTrack.Builder();

            audioTrack = builder.SetAudioFormat(new AudioFormat.Builder()
                .SetChannelMask(ChannelOut.Mono)
                .SetEncoding(Android.Media.Encoding.Pcm16bit)
                .SetSampleRate(provider.WaveFormat.SampleRate).Build()
            ).SetAudioAttributes(new AudioAttributes.Builder()
                .SetFlags(AudioFlags.LowLatency)
                .SetUsage(AudioUsageKind.VoiceCommunication)
                .SetContentType(AudioContentType.Speech).Build()
            ).SetBufferSizeInBytes(audioBuffer.Length).Build();

            audioTrack.Play();

            Thread playbackThread = new Thread(ReadLoop);
            playbackThread.Start();

            Logger.Info("[VOICE] Android Voice Output is ready");
        }
        void ReadLoop()
        {
            while(!doStop)
            {
                if(provider != null)
                {
                    if (doStop)
                        break;

                    int readBytes = provider.Read(audioBuffer, 0, audioBuffer.Length);

                    if (readBytes == 0)
                    {
                        Thread.Sleep(1);
                        continue;
                    }

                    if (doStop)
                        break;

                    audioTrack.Write(audioBuffer, 0, readBytes);

                    Thread.Sleep(1); // This didn't change anything whether in or not
                }
            }
        }

I get a mixture of massive 2s delay and sometimes the audio skips, which I assume is just where the internal audio buffer gets overwritten before I get a chance to pull from it again perhaps?

Is this something simple I'm missing? I'm fairly new to playing with streamed audio.

BenWoodford avatar Oct 17 '23 12:10 BenWoodford