trunk-recorder icon indicating copy to clipboard operation
trunk-recorder copied to clipboard

Some recordings leak into the wrong files

Open Nokoa opened this issue 4 years ago • 26 comments

It appears that sometimes I have recording files for one thing that also contain parts from other talkgroups.

Attached, one is a fire dispatch tg and the other is a utilities tg. Note that the fire recording also contains apart of the utilities recording from the other file.

Looking at the file name it seems as they were both transmitted very close to eachother, on the same frequencies. TDMA? 2recordings.zip

Nokoa avatar Feb 28 '20 02:02 Nokoa

Assuming you're on the latest version (commit e987fceb3c081fa834dbfcc3b44a2c3c48709560 ) of trunk-recorder, this is indeed still possible to happen and only occurs when there are rapid-frequency changes. This normally will occur when there's a high CPU load or high-load on the SDR device. The result is what you hear.

This issue was much worse in 2019, and was resolved (mostly) with the double decimation update.

I've not dug too deep into the code of how TR/OP25 deals/handles the Control Channel to Voice channels assigned to each talk group. But more or less, it-appears-to, or at least seems to be an issue where the SDR/Trunk Recorder code is sluggish to change the freq slice fast enough from what the CC says.

bctrainers avatar Feb 28 '20 02:02 bctrainers

Thank you for providing the above information. This has still been occurring semi frequently using latest version. Just wanted to let you know in case you been working on it further...

Nokoa avatar Jul 07 '20 05:07 Nokoa

Just an update, this is still occurring frequently when audio from a different transmission is added at the end of a different audio file. I am recording about 10 analog channels, and 2 sites of a P25 Phase 2 system.

Nokoa avatar Mar 13 '21 01:03 Nokoa

I am having the same issue starting a few months back. I have updated to the latest several times since then and it is still happening. I have tried a nooelec and airspy SDR, same issue. This is on a Lenovo desktop that runs about 50% cpu most the time and is only used for trunk-recorder, so I thought resources would be OK. This is on a P25 Phase 1 system, only digital recorders in use.

gonzoblue avatar Jul 16 '21 15:07 gonzoblue

Just adding a "me too" here. Happens with a Raspberry Pi 4 and RTL-SDR dongle, built from the latest Git checkout. Recording a P25 Phase 1 system, all digital.

ab0tj avatar Sep 01 '21 20:09 ab0tj

Could you give the 4.0-beta branch of the code a try? https://github.com/robotastic/trunk-recorder/tree/4.0-beta I re-wrote the call handling so it should strongly separate everything!

robotastic avatar Sep 02 '21 02:09 robotastic

Could you give the 4.0-beta branch of the code a try? https://github.com/robotastic/trunk-recorder/tree/4.0-beta I re-wrote the call handling so it should strongly separate everything!

I will give it a try in the next few days. Thank you for pushing this change

Nokoa avatar Sep 02 '21 13:09 Nokoa

Could you give the 4.0-beta branch of the code a try? https://github.com/robotastic/trunk-recorder/tree/4.0-beta I re-wrote the call handling so it should strongly separate everything!

Installed this morning. So far so good, but we'll see how the day plays out when it gets a little busier. The calls end up here: https://openmhz.com/system/codtrsbr Sorry about the occasional dropouts... A better antenna is on the way.

ab0tj avatar Sep 02 '21 14:09 ab0tj

I also switched to rtl_tcp on the Pi and Trunk Recorder is now running on a VM on a much more powerful machine in case it was a resource issue. That didn't seem to help with the 3.x branch.

ab0tj avatar Sep 02 '21 15:09 ab0tj

Thanks for providing this update. I have a version from a couple days ago and for the most part it works great. However I noticed a couple very small instances where audio from another transmission is heard at end of another talkgroup transmissions. I will look into it some more to confirm that it’s actually occurring.

Nokoa avatar Oct 16 '21 08:10 Nokoa

Thanks - if you are able to capture the output from trunk recorder when you are noticing this happen, that would help.

robotastic avatar Oct 16 '21 13:10 robotastic

I checked the files directly, and the one instance I noticed it happened I found, and can confirm that a couple of transmissions leaked into the same file of another talkgroup. Attached are photos of the file names, showing that the same frequency was used, at around the same time. In addition to a waveform of the audio from both files. The audio is in a different language so I will not be attaching it here. The 4087 talkgroup provides a single transmission by a single RID, in addition to multiple transmissions that are a short conversation between 2 RIDs from talkgroup 1825. The whole conversation in the file from 1825 is inside the 4087 file. This system is a P25 Phase 2 TDMA. These talkgroups do not have a patch or multi select with one another.

I did notice this a couple times, but this was the only one I directly found. Let me know if you want to know more.

tr-recs-dupes image0 (2)

Nokoa avatar Oct 16 '21 19:10 Nokoa

Thanks - Any chance you were also able to capture the log info the Trunk Recorder prints out when this overlap happened? That would help narrow down what is causing it.

robotastic avatar Oct 16 '21 20:10 robotastic

Unfortunately I don't think I have log saving on, I couldn't locate a log file. But I just turned it on, so hopefully I can get capture it next time.

On another note, I noticed that trunk-recorder is saving 2 audio files 1 wav and 1 m4a, not sure if I have something misconfigured or what not. I tried setting compressWav to false, and it didn't seem to change anything. This is an issue for me as a directory watcher from rdio-scanner was going through these folders and it would only delete the json and wav files, but I am left with all the m4a files.

Nokoa avatar Oct 16 '21 20:10 Nokoa

Disregard the note about the file types, it appeared I placed compressWav in the wrong place in the config, I placed it in the global stuff rather than in the specific system.

Nokoa avatar Oct 16 '21 20:10 Nokoa

This looks identical to the issue I'm experiencing all the time on my install (dupsc21 on openmhz) - I'll check the logs for a definitive overlapping call period and post details in this thread.

carissavixen avatar Oct 16 '21 23:10 carissavixen

Okay, found a call this afternoon that contains the glitch. Here's the logs of that call. Specifically, Unit ID 1106852 should not be in the TG 2607 file - that's a unit on a completely different talkgroup (TG 9011), which was recorded correctly in its own file. I also included that call's log. It looks like, at least for this one that I grepped out, these have sequential call log numbers log.txt .

carissavixen avatar Oct 17 '21 01:10 carissavixen

@carissavixen great job capturing this! It is really interesting -

[2021-10-16 19:52:02.368859] (info)   [dupsc21]	32682C	TG:       2607	Freq: 7.71106e+08	Starting new Transmission 	Src ID:  279923
[2021-10-16 19:52:14.392620] (info)   [dupsc21]	32682C	TG:       2607	Freq: 7.71106e+08	Starting new Transmission 	Src ID:  279923

That second New Transmission, looks like it is actually from 1106852 and should have not been started. I will go double check the logic and see if some more checks are needed... and add some more debug messages to see if I can catch anything.

robotastic avatar Oct 17 '21 12:10 robotastic

OK - I found a bug in how it was tracking updates from Freq allocation to TG aka Call Timeouts. Can you try this branch: debug-overlap

Look for the TG Mismatch messages.... this is a sign that a recorder has started to record something from a different talkgroup. If you see this still, can you try adding: "callTimeout": 2 to the config.json as part of the top level settings. If you still see that message, can you try changing it to: 1

robotastic avatar Oct 17 '21 15:10 robotastic

Ok, had to switch from a basic Docker install to a local build. I'm now running with the debug-overlap changes applied. I don't see TG Mismatch in the code changes, is that the right string?

carissavixen avatar Oct 17 '21 21:10 carissavixen

@carissavixen Sorry!! that is because I am an idiot and forgot to push the changes to github. https://github.com/robotastic/trunk-recorder/blob/8831c9bdddb540afcb8ba56b94a7f26e73f1f739/lib/gr_blocks/nonstop_wavfile_sink_impl.cc#L356

Look for TG MISMATCH!

robotastic avatar Oct 18 '21 01:10 robotastic

Ok, debug-overlap Take 2 is running - definitely seeing a lot of "TG MISMATCH", even with "callTimeout": 1 log_snippet.txt

carissavixen avatar Oct 18 '21 01:10 carissavixen

Ah! yep - I had it set to check against SmartNet talkgroups, and you have to shift left 4 times to get that number. I just changed it so it should check both SmartNet and P25 type talkgroup ids....

robotastic avatar Oct 18 '21 02:10 robotastic

Okay, running again: just a handful of TG MISMATCH when I first started up the new copy, haven't seen any again (yet, only running for about 5 minutes). I still have the timeout set to 1 for now.

carissavixen avatar Oct 18 '21 02:10 carissavixen

Is there any updates on this... s'Seeing this a lot on a smart zone smartnet system here... gets really bad when the system is busy.. let me know if there is any info i can provide to help

tdduarte avatar Feb 03 '22 14:02 tdduarte

@tdduarte If you are able to catch the log messages around when this is happening, that would be great. I thought I had mostly fixed this, but I could be missing something.

robotastic avatar Feb 04 '22 03:02 robotastic