Inconsistent volume in recorded calls

[theficus]

This may be similar to #677 but the symptoms I'm running into seem somewhat different.

Scenario: I’m scanning KCERS Site 040 which covers approximately 825.225M-826.1375M. I'm running into a problem that some recordings are very quiet while others are loud and clear. There doesn't seem to be any rhyme or reason as to why and there's no errors logged. It's almost as if sometimes the analogLevels value is being applied to boost the call volume, and sometimes it's not.

This inconsistent audio issue doesn't seem specific to any one RTL-SDR dongle or talkgroup. It's just that sometimes the call comes in loud and clear, and other times it's super quiet. It's also seems to be linear (when it's quiet, everything's quiet) so I don't think it's simply noise drowning out the call.

I upload all my calls to OpenMHZ here: https://openmhz.com/system/kcers040. If you go through calls this should be observable as sometimes they come in nice and clear with almost zero noise, and other times you can barely hear them.

tunk-recorder has been running pretty well -- it's been going for days now capturing over 10k calls without a single hiccup. There's the never-ending struggle of finding the perfect combination of squelch, gain, and other micro-adjustments but for the most part I think call quality is pretty decent. This inconsistent volume issue is probably one of the biggest annoyances right now.

If anybody has any thoughts on if this is a bug or something I can tweak in the config file I'll try anything. If there's any additional debugging or other information I can collect on this, I'm game as well.

Thanks!

Setup: Remtronix 820S antenna on an indoor magnetic base ➜ Nooelec Flamingo+ FM band pass filter ➜ Nooelec Lana LNA (USB powered) ➜ DC blocker ➜ Generic cable TV splitter ➜ Two Nooelec SMArt RTL-SDR dongles ➜ USB hub ➜ Raspberry Pi 4

Here's my redacted configuration file:

{
    "ver": 2,
    "frequencyFormat": "mhz",
    "defaultMode": "analog",
    "controlWarnRate": 10,
    "captureDir": "./captures",
    "controlRetuneLimit": 6,
    "callTimeout": 1,
    "logFile": true,
    "logLevel": "info",
    "debugRecorderx": true,
    "debugRecorderAddress": "127.0.0.1",
    "statusServer": "ws://127.0.0.1:3010/server",
    "broadcastifyCallsServer": "https://api.broadcastify.com/call-upload",
    "uploadServer": "https://api.openmhz.com",
    "audioStreaming": true,
    "sources": [
        {
            "center": 853559375,
            "rate": 2048000,
            "error": 0,
            "ppm": 0,
            "gain": 7.7,
            "analogRecorders": 2,
            "driver": "osmosdr",
            "device": "rtl=00000003"
        },
        {
            "center": 855675000,
            "rate": 2048000,
            "error": 0,
            "ppm": 0,
            "gain": 7.7,
            "analogRecorders": 2,
            "driver": "osmosdr",
            "device": "rtl=00000002"
        }
    ],
    "systems": [{
        "control_channels": [
            853650000,
            853900000
        ],
        "type": "smartnet",
        "bandplan": "800_reband",
        "squelch": -62,
        "minDuration": 0,
        "annalogLevels": 16,
        "modulation": "fsk4",
        "talkgroupsFile": "kcers-rr.csv",
        "talkgroupDisplayFormat": "tag_id",
        "broadcastifyApiKey": "x,
        "broadcastifySystemId": "x",
        "shortName": "kcers040",
        "apiKey": "x"
    }]
}

[theficus]

I boosted the gain up to 13 and dropped the squelch down to -63 and now all calls seem to be coming in pretty consistently loud and clear so far...

I'll keep a watch on this and if that's really all there was to it, I'll close this issue out.

[theficus]

Unfortunately, still seeing this.

Here's an example of two calls, same talkgroup, same frequency, right around the same time.

Call 1: https://openmhz.com/system/kcers040?call-id=62f1237cd1bf3687fc8c01e4&time=1659970421001 Call 2: https://openmhz.com/system/kcers040?call-id=62f123a5d1bf3687fc8c0a13&time=1659970459001

Note how call 1 is very loud and clear while call 2 is much quieter by comparison. It feels very linear -- as if no amplification is being applied to call 2. If it was only the call itself that was quiet, I'd expect the hiss to be just as loud between the two, but even that's quieter.

[robotastic]

hmmm... There isn't any sort of AGC in the analog recorder decode path. The weird thing is that the sox command didn't do a better job of normalizing that quiet one. I would try playing around with the sox parameters and see if the is a better setting. It is totally possible that it is messing up the volume levels: https://github.com/robotastic/trunk-recorder/blob/a30ee308518d478c4fdcd39fab2314717898bd65/trunk-recorder/call_concluder/call_concluder.cc#L31

[taclane]

Plotting the two in audacity, it does look like this is just sox doing its usual "fast but not very good" normalization pass. Sox is adjusting the volume of the entire call up or down depending on the single loudest point it finds. In digital modes, it's usually wind or road noise that leads to inconsistent audio, but in analog the likely culprit is a loud squelch tail. In the second call, you can see how the squelch burst at the end was much louder than the voice audio. Normalization couldn't bring the volume of the second call any higher than where that spike hits just below 1.0 on the scale (the --norm=-.01 dBFS in the sox command).

There may be ways to tune out those bursts better with a sox compand command, or just maybe run it through ffmpeg with the loudnorm or speechnorm plugin. Over in the discord, I don't think there's been a consensus on what the best method is to address inconsistent audio levels in recordings.

[theficus]

Thanks for this analysis -- it makes a lot of sense.

I just started looking at this code last night after @robotastic pointed to it. I haven't messed with sox in at least a decade so I'm trying to reorient myself here.

When I get some time I'll play around with this some more to see if I can find a good solution. I'll create a PR if I make any breakthroughs.

(I guess as a related feature request, it would be really nice if this all could be manipulated in the json rather than requiring recompilation.)

[rosecitytransit]

If you're still having the issue and the squelch tail is the problem, one option would be to cut off the very end of the recording, presuming you can consistently do so without losing any actual talking.