sdrtrunk icon indicating copy to clipboard operation
sdrtrunk copied to clipboard

MP3 stream corruption

Open amigan opened this issue 2 years ago • 10 comments

Using the latest HEAD cbeda19d27643910d0a15f8bb6f52e7425609325, I have noticed occasional "skipping" and garbling of the MP3 stream going through my icecast server. I also see errors when listening with mpv:

% mpv http://scanner/riscon
 (+) Audio --aid=1 (mp3 1ch 8000Hz)
AO: [pulse] 8000Hz mono 1ch float
A: 00:00:07 / 00:00:35 (21%) Cache: 27s/254KB
[ffmpeg/audio] mp3float: big_values too big
[ffmpeg/audio] mp3float: Error while decoding MPEG audio frame.
Error decoding audio.
A: 00:00:07 / 00:00:35 (21%) Cache: 27s/253KB
File tags:
 icy-title: Scanning...
A: 00:00:23 / 00:00:46 (50%) Cache: 22s/206KB
[ffmpeg/audio] mp3float: big_values too big
[ffmpeg/audio] mp3float: Error while decoding MPEG audio frame.
Error decoding audio.
A: 00:01:25 / 00:01:25 (100%) Cache: 0.0s

Device is RTL-SDR, system is P25 phase I. Built and running on a Raspberry Pi 4 running Manjaro/arm64 and liberica-jdk-full.

This was not occurring with SDRTrunk as of 18 February HEAD. I am happy to perform further troubleshooting or provide any requested information.

amigan avatar May 01 '22 16:05 amigan

I'm guessing by the URL that it's a self-hosted icecast server... What package/version is it?

Can you try going into the stream options and disabling "inline metadata"?

doug-hoffman avatar May 01 '22 19:05 doug-hoffman

It is indeed my own icecast instance, on the same LAN. It's icecast-kh-2.4.0.k15 running on FreeBSD. Disabling inline metadata fixed mpv so that it actually shows metadata now and no longer emits errors, but the audio is still garbled.

amigan avatar May 02 '22 02:05 amigan

I can confirm that inline metadata is broken with 2.4.0-kh15. It works fine with 2.3.2-kh32 (what Broadcastify uses) and 2.4.4, so it appears to be a bug in the 2.4.0-kh fork.

No issues with audio though, other than those caused by 2.4.0-kh failing to process inline metadata that result in metadata getting decoded as audio.

(Edit 1: I only tested on x86_64 platforms though. No spare arm stuff that I can test with.)

(Edit 2: Inline metadata works fine with 2.4.0-kh1, so it's definitely broken by a change in the kh fork.)

doug-hoffman avatar May 02 '22 11:05 doug-hoffman

The latest KH commit has a lot of "allow for setting a different default charset to utf8" messages, so maybe you must explicitly set the character set to avoid munging the metadata?

boteman avatar May 02 '22 20:05 boteman

Interesting find. I rebuilt kh15 with "ICY_CHARSET=latin1" and am waiting for some radio traffic to see if it made any difference.

doug-hoffman avatar May 02 '22 20:05 doug-hoffman

Nope...

$ mpv http://127.0.0.1:8000/stream
 (+) Audio --aid=1 (mp3 1ch 16000Hz)
AO: [coreaudio] 16000Hz mono 1ch floatp
A: 00:00:07 / 00:00:11 (65%) Cache: 3.9s/61KB
File tags:
 icy-title: 
A: 00:00:15 / 00:00:23 (69%) Cache: 6.9s/111KB
File tags:
 icy-title: Scanning...
A: 00:08:27 / 00:08:52 (95%) Cache: 24s/398KB
[ffmpeg/audio] mp3float: invalid block type
[ffmpeg/audio] mp3float: Error while decoding MPEG audio frame.
Error decoding audio.
A: 00:08:32 / 00:08:54 (96%) Cache: 22s/358KB
[ffmpeg/audio] mp3float: invalid new backstep -1
A: 00:08:44 / 00:09:03 (97%) Cache: 18s/298KB
[ffmpeg/audio] mp3float: big_values too big
[ffmpeg/audio] mp3float: Error while decoding MPEG audio frame.
Error decoding audio.
A: 00:08:49 / 00:09:07 (97%) Cache: 17s/280KB

doug-hoffman avatar May 02 '22 21:05 doug-hoffman

Digging into the source a little further, source_mp3->inline_metadata_interval is the interval from the source and source_mp3->interval is the interval to downstream clients. It should still parse incoming metadata at the correct interval.

It's possible to override the outgoing metadata interval in the icecast configuration to work around the limit:

<mount type="normal">
    <mount-name>/stream</mount-name>
    <mp3-metadata-interval>2000</mp3-metadata-interval>
</mount>

The metadata works for a while, but eventually gets scrambled. The cause appears to be a transition from a long title to a short one:

[2022-05-02  20:44:19] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Scanning...';...
[2022-05-02  20:44:21] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:21] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:22] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:23] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:44] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';ning...';...
[2022-05-02  20:44:51] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUStreamTitle='Scanning...';...
[2022-05-02  20:45:37] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';e='Scanning...';...
???X??05-02  20:48:18] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';!k??(?4
      O"@%StreamTitle='TO:200...
[2022-05-02  20:48:20] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUUUUStreamTitle='Scanning...';...
[2022-05-02  20:48:27] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU...

I have confirmed with packet captures that we are still sending valid metadata.

It will work seemingly forever if the metadata remains as 'Scanning...', but as soon as it changes to something else and back, things go wrong.

It's also worth noting that if you let it run long enough, it'll make the icecast daemon completely unresponsive. Probably something vuln hunters would like to spend some time with.

doug-hoffman avatar May 03 '22 01:05 doug-hoffman

2.4.0-kh10 appears to be the last version without the problem. Changes in kh11: https://github.com/karlheyes/icecast-kh/compare/icecast-2.4.0-kh10...icecast-2.4.0-kh11

doug-hoffman avatar May 03 '22 20:05 doug-hoffman

I've created karlheyes/icecast-kh#368 to track this issue in icecast-kh.

doug-hoffman avatar May 03 '22 20:05 doug-hoffman

Thank you for your sleuthing. When I return home from work tonight I will try to see if I can dig deeper on troubleshooting the audio issue.

amigan avatar May 04 '22 15:05 amigan

This issue is fixed by #1244. Currently, there is extraneous data being streamed from each MP3 file which results in icecast-kh getting out of sync with the metadata position. The new behavior only sends valid frames which doesn't trigger the bug in -kh.

doug-hoffman avatar Dec 17 '22 21:12 doug-hoffman

Closing - #1244 PR merged.

DSheirer avatar Dec 18 '22 10:12 DSheirer