KejPi
Postprocessing script for Scanning Tool CSV files
Hi all,
I use this script to help collect DX logs for FMList (specifically the DABList section). I usually let my bandscan run overnight during expected tropo conditions (according to DXInfocentre), but going through the CSV output manually takes a lot of time - especially gathering all channels, locations, signal strengths, and first/last reception times.
So I wrote a Python script to do the heavy lifting and output the relevant data in a clean format.
Here’s a sample of what the output looks like:
6A, 6A OVL, 26.4 dB, Markelo/Cellnex toren, 34.3 km, 11:37 - 13:41
6A, 6A OVL, 26.4 dB, Zwolle/NOVEC mast, 2.4 km, 11:37 - 13:41
6D, 6D Zwolle, 30.1 dB, Zwolle/NOVEC mast, 2.4 km, 11:37 - 13:37
7C, 7C Gron / Drent, 4.7 dB, Hoogersmilde/Cellnex toren, 51.4 km, 11:38 - 13:37
7D, MTVNL, 18.3 dB, Lelystad/Cellnex toren, 46.4 km, 11:38 - 13:37
7D, MTVNL, 18.3 dB, Markelo/Cellnex toren, 34.3 km, 11:38 - 13:37
8B, 8B N-H / Flevo, 4.3 dB, Lelystad/Cellnex toren, 46.4 km, 11:39 - 11:55
8B, 8B N-H / Flevo, 4.2 dB, Lelystad/Cellnex toren, 46.4 km, 12:27 - 13:21
9A, WDR Regional, 4.5 dB, Münster/Baumberge, 101.8 km, 11:39 - 13:32
9C, 9C, 29.3 dB, Markelo/Cellnex toren, 34.3 km, 11:39 - 13:38
9C, 9C, 29.3 dB, Zwolle/NOVEC mast, 2.4 km, 11:39 - 13:38
10A, NDR NDS OS, 6.4 dB, Lingen/Damaschke, 83.9 km, 11:40 - 13:38
10B, 10B Steden3hoek, 10.8 dB, Apeldoorn/Toren Noord, 27.7 km, 11:40 - 13:39
10B, 10B Steden3hoek, 10.8 dB, Deventer/Wezenland, 22.8 km, 11:40 - 13:39
11C, DAB+, 30.4 dB, Markelo/Cellnex toren, 34.3 km, 11:40 - 13:39
11C, DAB+, 30.4 dB, Zwolle/NOVEC mast, 2.4 km, 11:40 - 13:39
11D, WDR NRW, 6.9 dB, Schöppingen, 88.4 km, 11:57 - 13:07
11D, WDR NRW, 6.3 dB, (unknown), km, 11:41 - 13:40
12B, 12B UTR / GLD, 29.0 dB, Markelo/Cellnex toren, 34.3 km, 11:41 - 13:40
12B, 12B UTR / GLD, 29.0 dB, Ugchelen/Cellnex mast, 37.2 km, 11:46 - 13:40
12B, 12B UTR / GLD, 29.0 dB, Zwolle/NOVEC mast, 2.4 km, 11:41 - 13:40
12C, NPO, 28.2 dB, Ugchelen/Cellnex mast, 37.2 km, 11:47 - 13:02
12C, NPO, 27.7 dB, Ugchelen/Cellnex mast, 37.2 km, 13:40 - 13:40
12C, NPO, 28.3 dB, Zwolle/NOVEC mast, 2.4 km, 11:41 - 13:40
12D, Flevoland, 5.0 dB, Lelystad/Cellnex toren, 46.4 km, 11:47 - 12:19
12D, Flevoland, 4.8 dB, Lelystad/Cellnex toren, 46.4 km, 13:03 - 13:41
This makes entering logbook entries into FMList a lot faster and easier. I’ve attached the Python script in case it’s useful for anyone else!
Anything I can implement in application to help you?
Thank you so much! The app is already fantastic - I use it continuously.
One feature that would really help me is a "mark as known" function in the Scanning Tool. Let me try to explain:
FMList distinguishes between continuous (normal/local) reception and incidental (tropo/DX) reception. As a user, I need to enter continuous reception in the Bandscan section, and DX/tropo reception in the Logbook.
What would be incredibly helpful is a way to mark a channel/transmitter combination as "known", perhaps via a right-click in the Scanning Tool results list. Then, in the Scanning Tool interface, a checkbox like "hide known" could filter those out - so I only see newly detected (potential DX) results.
Do you think this is a good idea, or do you have an even better solution?
What about excluding those known channels from the scan?
That’s a good thought, but the challenge is that many channels carry signals from multiple transmitters. For example, here in the Netherlands, channel 12C already has a long list of transmitters: https://radio-tv-nederland.nl/dab/dab-overzicht-12c.html
Under normal conditions, I only receive 3 of those. But during tropo, I can sometimes receive up to 28 different transmitters on that same channel.
At the moment, there's no way to filter at the transmitter level - only by channel. So a full bandscan produces a very long list.
In this screenshot, you can see many channels that are normally empty here - I had deselected most of the "known" ones before going to bed that night. But the next morning, when I checked the reception logs of other users, I saw I had missed a lot of DX reception on known channels, because the extra transmitter locations were filtered out.
This is my list during normal reception 😃
OK, so the point is to exclude individual transmitters rather than channels. I think it is not a technical problem to exclude something, the problem is how to integrate this feature in UI also considering a possibility to remove something from excluded list. I will think about it.
OK, so the point is to exclude individual transmitters rather than channels. I think it is not a technical problem to exclude something, the problem is how to integrate this feature in UI also considering a possibility to remove something from excluded list. I will think about it.
Correct. Excluding individual transmitters of a channel. Thank you for taking it into consideration! Just let me know if you need a tester.
Please also note to quote text, as I don't know if we have commas in Location or Ensemble Name.
For those interested, I’ve created a second script that can be used in combination with the first script (dab_processor.py) to filter DAB reception results. This saves you from too much thinking in the early mornings... 😉
You can pipe the output from the first script into this one to automatically exclude known reception entries. I use ./dab_processor.py | ./filter_new_reception.py.
The script uses a separate text file (known_reception.txt) where you can list known permanent receivable local channel/transmitter location combinations. Only new (previously unknown) reception entries will be printed - perfect for identifying what should go into the FMList DAB Logbook, while known stations should already be in your the FMList Bandscan.
Included:
- The filter script (filter_new_reception.py)
- A sample known_reception.txt
Notes:
- Malformed lines are silently skipped.
- Location matching is case-insensitive and substring-based - this allows for partial matches like "Zwolle" to match "Zwolle/NOVEC mast".
- Ensure known_reception.txt is located in the same directory as the script.
OK, so the point is to exclude individual transmitters rather than channels. I think it is not a technical problem to exclude something, the problem is how to integrate this feature in UI also considering a possibility to remove something from excluded list. I will think about it.
Correct. Excluding individual transmitters of a channel. Thank you for taking it into consideration! Just let me know if you need a tester.
I assume that you want to have a possibility to export to CSV all records or to export only those that are not marked as known when known are filtered out, correct?
OK, so the point is to exclude individual transmitters rather than channels. I think it is not a technical problem to exclude something, the problem is how to integrate this feature in UI also considering a possibility to remove something from excluded list. I will think about it.
Correct. Excluding individual transmitters of a channel. Thank you for taking it into consideration! Just let me know if you need a tester.
I assume that you want to have a possibility to export to CSV all records or to export only those that are not marked as known when known are filtered out, correct?
Exactly. That's correct! 😃
OK, so the point is to exclude individual transmitters rather than channels. I think it is not a technical problem to exclude something, the problem is how to integrate this feature in UI also considering a possibility to remove something from excluded list. I will think about it.
Correct. Excluding individual transmitters of a channel. Thank you for taking it into consideration! Just let me know if you need a tester.
Please contact me on my mail (in About), if you still want to be tester.