xiph
Large volume burst & clipping when encoding a sine sweep at 32kbps
Found this from this story, while looking at the low-quality encode from YouTube. To reproduce, grab this file and:
opusenc --bitrate 32 sweep-noise.wav sweep-noise.opus
The original is a constant-amplitude sine sweep (ignoring the noise part in the second half), but this is what opusenc spits out:
There is both an explosion at ~1.5k - 2k and fading at ~8k - 8.5k, as well as some pretty visible aliasing mirroring about 8kHz:
I assume the upper bands being just noise is normal behavior at this bitrate to preserve energy; I don't think the explosion and fading are though. The aliasing, I'm guessing, is an artifact of the SILK/CELT crossover filter not being steep enough? (which is probably an unrelated issue)
Note: the .wav is decoded from a higher quality Opus, which is why it's not perfectly clean. I don't have the original sweep, but this sample reproduces the problem just as well, so it doesn't matter.
Version: opusenc opus-tools 0.2 (using libopus 1.3.1-119-g3acaa70965c5)
Also happens with the old 1.3.1 release, though possibly not the fading (that part might be a regression, but the explosion is in both).
The FFmpeg encoder doesn't explode, although it drops into the noise bands faster. No aliasing too. It doesn't do as good a job on the subsequent noise section though (it sounds chirpy, like libopus just gives up and encodes the bands as noise while ffmpeg tries to actually encode signal and makes it worse).
Setting OPUS_APPLICATION_RESTRICTED_LOWDELAY yields near "perfect" output (for 32k, anyway):
While OPUS_APPLICATION_VOIP and OPUS_APPLICATION_AUDIO both reproduce the problem, with slight differences in the length of the explosion. That suggests this is a SILK issue, and maybe the noise bands are a bit allocation problem where CELT doesn't have enough bits to cleanly code the sine sweep (or doesn't think it does)?
Can you share your .opus file? I'm unable to replicate the explosion artifact on a near-latest Opus git (I get all other artifacts though):
My opus file (gzip-encoded for GitHub): sweep-noise-gcc-o2-float.opus.gz
^ The spectrogram above is from a custom-compiled x86-64 gcc optimized float build (libopus 1.3.1-142-gcaf56aab, libopusenc 0.2.1-14-gf51c3aa-dirty). I also tried a clang debug fixed-point build (sweep-noise-clang-o0-fixed.opus.gz) and the stock opus-tools that comes with Fedora 35 (sweep-noise-fedora-35.opus.gz) with the same results. Also got the same results after ripping the 32kbps .opus directly from the YouTube video.
Side-note: I find it really odd the "aliasing" artifact is extending backwards in time (it makes an X shape instead of a ʎ shape), like there's some kind of frequency shift and it's actually aliasing about 6kHz (not 8kHz) then shifted up by 2kHz.
And you're right - opusdec decodes it properly. So it looks like what I'm hitting is a bug in the ffmpeg opus decoder!
I guess that means we can close this. Sorry for the noise!
Which version of opusdec did you test against? Because with my own code (which I cannot share in full, but if required I can extract the opus encoder and decoder parameters we use) I can still reproduce this bug.
(edit: just to clarify, that's three sine sweeps in a row, and they should be all identical to the first one)
Not sure if it's exactly the same issue as described here, but using latest master (bce1f392353d72d77d543bb2069a044ae1045e9d) and the included opus_demo tool as follows:
./opus_demo -e audio 48000 1 32000 -complexity 8 sweep.wav sweep.opus
./opus_demo -d 48000 1 -loss 1 .\sweep.opus result.raw
Using this test file: sweep.zip
I get this result:
The simulated 1% packet loss on decode is what is causing it in this particular case.