WP4000 - Controlled OTA Testing

[drowe67]

We'd like to find out how our prototype new waveform performs compared to (a) earlier FreeDV waveforms (b) SSB (c) on a range of channels. We'd like to evaluate performance as early as possible, as the next stages of C porting and FreeDV integration involve quite a lot of work. We need to know if we are on the right track.

In this WP we will transmit the new waveform over real HF channels, at the same time as samples of SSB and earlier waveforms, then evaluate the results.

• [ ] Consider splitting this into several WP (it's getting big)
• [ ] Decide what the metrics are - e.g. PER or actual Codec 2 voice? Any acquisition stats we want to measure?
• [ ] Use Octave implementation of modem from #18
• [ ] Transmit candidate waveform(s) OTA
• [ ] Concatenate with control waveforms (e.g. 700D, 700E, SSB) so they are sent at approx the same time, which will mean the same channel.
• [ ] Comply with station ID rules, e.g, have a station ID header, try not to Tx on top of any other stations (can we listen before we Tx?)
• [ ] Script based, e.g. ota* at https://github.com/drowe67/codec2-dev/tree/master/unittest was used for similar testing, write up here Controlled FreeDV Testing
• [ ] Design so the experiment can be repeated by other Hams, in order together data from different geographical locations. This means some documentation on setup, running, common issues. Perhaps reception by receivers other than KiwSDRs
• [ ] Design so we can accept manually collected samples.
• [ ] Consider (perhaps another WP) repeating experiment at different locations to capture any local issues, for example run in Japan and the UK to evaluate performance on NVIS channels common in those locations.
• [ ] Test over HF, LEO, GEO channels
• [ ] HF channels should include NVIS, 800-2000km, and multihop international (half way around globe), and polar paths. Can we break the modem on any channel?
• [ ] Rx by one of more (Kiwi) SDRs
• [ ] Automate, run over 1 week, 100s of samples
• [ ] Collect data, measure PER, process, present for interpretation
• [ ] Code (and data?) in it's own repo or sub-dir (not codec2/)

Deliverables:

• [ ] code, documentation on how to set up and run in README, results.
• [ ] Write up on Gdoc and/or FreeDV blog: summary of results, discussion, conclusions, further work, modem issues found (that need to be fixed)
• [ ] Reviewed