FullSubNet-plus icon indicating copy to clipboard operation
FullSubNet-plus copied to clipboard

small code fix needed in clipping detection

Open ariel-c-verbit opened this issue 2 years ago • 3 comments

Hey, first of all I'd like to thank you for this great model and for sharing it on github! a small bug i found:

as we know, cIRM isn't bounded and thus we are able to get mask amplitudes that are larger than 1. this can cause clipping in the enhanced signal.

to fix this you check it:

if abs(enhanced).any() > 1: print(f"Warning: enhanced is not in the range [-1, 1], {name}")

I think you meant:

if (abs(enhanced) > 1).any(): print(f"Warning: enhanced is not in the range [-1, 1], {name}")

after fixing this I see quite a lot of clipping...

https://github.com/hit-thusz-RookieCJ/FullSubNet-plus/blob/a6c89083cd083e729ca3def9a291743e8c3b516b/speech_enhance/audio_zen/inferencer/base_inferencer.py#L148

ariel-c-verbit avatar Aug 03 '22 03:08 ariel-c-verbit

First of all, thank you very much for your correction on this code! I will fix this part later. Secondly, I don't quite understand what you said about clipping, because the code subsequently normalizes the enchanced. Will this cause a clipping problem? enhanced = np.int16(0.8 * amp * enhanced / np.max(np.abs(enhanced)))

RookieJunChen avatar Aug 03 '22 03:08 RookieJunChen

What causes signal values that are out of range [-1,1] is the masking itself. (cIRM amplitudes larger than 1). This is due to the model and is a common cIRM problem.

If I understand correctly, you normalize the enhanced signal to [-0.8,0.8], thus avoiding this problem.

This fixes it, but this isn't a solution that can be used in a real life live system. (for academia it is good enough though).

In a real life system (where you receive frames and not the full signal), you would need an adaptive gain control normalization or something like that. You can't just assume that you have the whole signal in advance and normalize every couple of frames to [-0.8, 0.8] since that would amplify the noise in some cases. hope this helped.

ariel-c-verbit avatar Aug 03 '22 04:08 ariel-c-verbit

Thank you very much for your response, it has been quite helpful to me!

RookieJunChen avatar Aug 03 '22 04:08 RookieJunChen