macOS/Linux: Heavy audio distortion with v3 NDI audio APIs

[PatTheMav]

Description

When running obs-ndi off the master branch, audio sent by any NDI source (tested with OBS as well as NDI's own apps) will be received with heavy digital aliasing (aka "very crunchy").

Steps to Reproduce

• Run an NDI source
• Add NDI source to OBS using obs-ndi built from master branch
• Add monitoring to NDI source
• Hear crunchy audio

Additional comments

I did a bisect on all commits and found that this commit introduced the issue:

https://github.com/Palakis/obs-ndi/commit/7b04e64b12821d2dcbc168c79694119c707306e3

I dunno what it is about NDIlib_audio_frame_v3_t that mangles the audio, but all commits before this one have proper audio.

This also confirms the findings in https://github.com/CoSMoSoftware/OBS-studio-webrtc/issues/292 - mainly that reverting to version 4.9.0 fixes it and that the "sizzling" only happens on the right channel, while the left channel is clean.

[PatTheMav]

My bad - obvious duplicate to https://github.com/Palakis/obs-ndi/issues/574 - leave it to you which one to close as duplicate.

[PatTheMav]

~~I might have fixed this by changing this single line:~~

See below.

[rebb]

Not sure if i misunderstand the code, but this also seems to help with the problem :

obs_audio_frame.data[i] = (uint8_t*)( &audio_frame.p_data[ i * audio_frame.channel_stride_in_bytes ]);

I really wish someone with a deeper understanding of the code was active in the issues section, to comment on these things :( But judging by the state of replies, it seems we're kinda alone in here.

[PatTheMav]

Yeah I was a bit stupid, my bad - so obs-ndi seems to use a channel stride of samples * 4 and uses that to identify the size for each channel.

So for decoding if you treat p_data as an array of type uint8_t then indices map to bytes and you can use the stride to calculate the index for each channel. So to me your fix looks like the correct one.

[rebb]

Not stupid at all, i guess most of us are just poking at things trying to fix stuff, because there's little maintainer feedback at the moment. And you tracked the problem down to that bit of code in the first place :+1:

[PatTheMav]

I guess Palakis was busy with the new obs-websocket version which came out in February, he also wrote the new track matte transitions functionality in OBS 27, so he's definitely active - all projects around OBS are kinda swamped with work right now and most of got day jobs to take care of as well, so at some point the brain just needs to take a big step back.

[rebb]

For whatever reason, using .channel_stride_in_bytes in a different context actually causes problems, and .no_samples is the correct way there.

So maybe my version is wrong after all - or it only applies when using certain functions to get the data ?

Back to confused poking around i guess.

[PatTheMav]

What is the different context? Because for NDIlib_FourCC_audio_type_FLTP (planar floating point audio) type audio streams the channel stride seems to be the right choice:

	union
	{	// If the FourCC is not a compressed type and the audio format is planar,
		// then this will be the stride in bytes for a single channel.
		int channel_stride_in_bytes;

		// If the FourCC is a compressed type, then this will be the size of the
		// p_data buffer in bytes.
		int data_size_in_bytes;
	};

IIRC OBS uses 32-bit floating point audio internally, which means 4 bytes per sample and as I mentioned obs-ndi uses num samples * 4 to calculate the stride on the encoding end.

So for 2 channels with 4 samples per frame that'd be 32 bytes of data, the channel stride would be 16 bytes accordingly.

[rebb]

I was experimenting with the frame-sync code from the @aquarat fork ( https://github.com/aquarat/obs-ndi ) and kind of blindly replaced the .no_samples code there, which causes OBS to crash.

I haven't really investigated much further as to why though, i just assumed that the framesync-capture does something different under the hood.

[mikeev261]

Anyone still looking into this? Or have a workaround? I'm getting a lot of crackling with any NDI source... using obs-studio and obs-ndi both compiled on an M1 Mac, #482. I assume it's this bug. It's dominant in the right channel, for whatever that's worth.

[mikeev261]

Here's a sample of what I'm hearing: https://www.twitch.tv/videos/1123610930

[amatheo]

Have any workaround has been found ? Same issue while running both obs and ndi compiled for arm64 on M1 Mac like @mikeev261

[mikeev261]

Have any workaround has been found ? Same issue while running both obs and ndi compiled for arm64 on M1 Mac like @mikeev261

The only workaround that I've found that actually works is to roll-back to a 4.9.0 compile. The bug was introduced in 4.9.1 (I believe).