MMDVMHost
MMDVMHost copied to clipboard
g4klx
Support for D-Star DV Fast Data
I've been testing D-Star DV "Fast Data" as implemented on the Kenwood TH-D74. I tried sending data via a ZUMspot only to find that it gets corrupted on its way through (https://github.com/juribeparada/MMDVM_HS/issues/71 has an example of the corruption). MMDVMHost also reports a BER of ~12.6%.
I discovered that if I comment out all three calls to m_fec.regenerateDStar() in DStarControl.cpp, the data is able to pass through without corruption. It seems that all 72 bits of each voice frame are used for data and there is no FEC applied, which also probably explains the ~12.6% BER report.
Some things I don't know:
- Is there a way to tell when a voice frame contains "fast data" instead of voice+FEC?
- Do Icom and Kenwood implement the same fast data spec? (I hope so)
- Why is the FEC recalculated and reapplied in MMDVMHost? Is it so that the BER reports are effectively per-hop? Is there something important to the protocol going on?
To answer question 1, I'm planning to look for a difference in the packet that indicates whether voice or data is contained within the voice frames. I think there must be a way to tell the difference, because the demos I've seen of Icom radios doing fast data allow the user seamlessly transmit audio while a data transfer is ongoing and the radio degrades to slow data while sending voice frames. (The Kenwood does not seem to have this slow data capability)
I'm going to email Kenwood and Icom in an effort to answer question 2 (and maybe question 1?) and also ask if they have any specs to share.
I'm hoping that someone reading this can answer question 3 for me.
I'm interested in all this because I think that fast data + hotspots and the reflector system could be a powerful combination.
-TimS, KG4BXH
Hi Tim
Fast data on normal D-Star radios is a relatively new thing and certainly wasn’t around when my software was first written. The only data mode being used then was the relatively high speed DD mode on 23cms.
Although I own two radios that do this mode, an Icom ID-51 and a Kenwood TH-D74, I have not tried it out. So my comments are based on understanding the protocol and code rather than direct experience of the mode.
So to your questions:
- I don’t know. There is a bit in the header that is marked as being to indicate whether the payload is data or voice. This is not checked in my software currently. I hope that they use this bit for this mode also, although they’re not the same structure. The flags are the first three bytes of the D-Star header. I think any dump will need to be at a low level, i.e. the raw serial data from the modem so that any patterns in the data can be observed.
- I don’t know.
- The FEC is recalculated so that the bit errors don’t accumulate. This is something that Icom repeaters don’t do, and almost all repeaters for other modes do. I am certainly not willing to remove it as it can make a big difference to the received audio, and voice mode is the vast majority of the traffic. If we can find a way to reliably detect this data mode, then we can do something about the FEC regeneration.
To create the traces you need to go into the [Modem] section of MMDVM.ini and set either Debug=1 or Trace=1 (or both, I forget, and I’m at work currently) and logging to level 1 for debug messages. You shouldn’t need much data.
There must be a way to distinguish the voice and data modes, so I await your data logging with interest. Jonathan G4KLX
Sent from Yahoo Mail for iPhone
On Wednesday, November 7, 2018, 02:39, Tim Stewart [email protected] wrote:
I've been testing D-Star DV "Fast Data" as implemented on the Kenwood TH-D74. I tried sending data via a ZUMspot only to find that it gets corrupted on its way through (juribeparada/MMDVM_HS#71 has an example of the corruption). MMDVMHost also reports a BER of ~12.6%.
I discovered that if I comment out all three calls to m_fec.regenerateDStar() in DStarControl.cpp, the data is able to pass through without corruption. It seems that all 72 bits of each voice frame are used for data and there is no FEC applied, which also probably explains the ~12.6% BER report.
Some things I don't know:
- Is there a way to tell when a voice frame contains "fast data" instead of voice+FEC?
- Do Icom and Kenwood implement the same fast data spec? (I hope so)
- Why is the FEC recalculated and reapplied in MMDVMHost? Is it so that the BER reports are effectively per-hop? Is there something important to the protocol going on?
To answer question 1, I'm planning to look for a difference in the packet that indicates whether voice or data is contained within the voice frames. I think there must be a way to tell the difference, because the demos I've seen of Icom radios doing fast data allow the user seamlessly transmit audio while a data transfer is ongoing and the radio degrades to slow data while sending voice frames. (The Kenwood does not seem to have this slow data capability)
I'm going to email Kenwood and Icom in an effort to answer question 2 (and maybe question 1?) and also ask if they have any specs to share.
I'm hoping that someone reading this can answer question 3 for me.
I'm interested in all this because I think that fast data + hotspots and the reflector system could be a powerful combination.
-TimS, KG4BXH
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Jonathan,
Now it totally makes sense why you would want to regenerate the FEC at each hop. Even though "error correction" is in the name, I was thinking of it as a mere checksum. I suppose it is lucky that the currently-installed Icom repeaters don't recalculate FEC, at least from the perspective of using fast data through them. I certainly don't have the experience to know how much it would improve repeater use if the repeaters also did the recalculation.
I will grab some traces and post the results here. Thank you for the quick guide. If I can find the flag, I should be able to put together a patch that disables the recalc only in the case that fast data is in use.
Here are my findings so far.
In fast data mode, I've found that the TH-D74 sends the following bytes in the first 21 voice frames before starting to send the actual data:
EE C2 A1 C8 42 6E 52 51 C3
After the 21st frame, the contents of the voice frames begin to vary, presumably with the actual data I tried to send. Indeed, even when I send a single A followed by a LF, the same 21 frames appear.
Here is a trace from the log that was generated by a D-RATS transmission as it started up and advertised that I was online:
D: 2018-11-09 05:06:49.213 RX D-Star Header
D: 2018-11-09 05:06:49.213 0000: E0 2E 10 40 00 00 4B 47 34 42 58 48 20 47 4B 47 *[email protected] GKG*
D: 2018-11-09 05:06:49.214 0010: 34 42 58 48 20 42 43 51 43 51 43 51 20 20 4B 47 *4BXH BCQCQCQ KG*
D: 2018-11-09 05:06:49.214 0020: 34 42 58 48 20 20 44 37 34 20 19 8A 00 2F *4BXH D74 .../*
D: 2018-11-09 05:06:49.214 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
M: 2018-11-09 05:06:49.215 D-Star, received RF header from KG4BXH /D74 to CQCQCQ
M: 2018-11-09 05:06:49.221 Debug: Mode set to D-Star
D: 2018-11-09 05:06:49.232 RX D-Star Data
D: 2018-11-09 05:06:49.232 0000: E0 11 11 EE C2 A1 C8 42 6E 52 51 C3 55 2D 16 00 *.......BnRQ.U-..*
D: 2018-11-09 05:06:49.232 0010: 2F */*
D: 2018-11-09 05:06:49.232 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2018-11-09 05:06:49.249 RX D-Star Data
D: 2018-11-09 05:06:49.249 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 30 1B FA *.......BnRQ.0..*
D: 2018-11-09 05:06:49.271 RX D-Star Data
D: 2018-11-09 05:06:49.271 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 1D 6F FA *.......BnRQ..o.*
D: 2018-11-09 05:06:49.288 RX D-Star Data
D: 2018-11-09 05:06:49.289 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 31 21 B3 *.......BnRQ.1!.*
D: 2018-11-09 05:06:49.312 RX D-Star Data
D: 2018-11-09 05:06:49.313 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 3D 2E FD *.......BnRQ.=..*
D: 2018-11-09 05:06:49.329 RX D-Star Data
D: 2018-11-09 05:06:49.329 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 32 27 F2 *.......BnRQ.2'.*
D: 2018-11-09 05:06:49.351 RX D-Star Data
D: 2018-11-09 05:06:49.351 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 04 3B F2 *.......BnRQ..;.*
D: 2018-11-09 05:06:49.373 RX D-Star Data
D: 2018-11-09 05:06:49.374 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 33 21 B3 *.......BnRQ.3!.*
D: 2018-11-09 05:06:49.391 RX D-Star Data
D: 2018-11-09 05:06:49.393 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 50 6F B3 *.......BnRQ.Po.*
D: 2018-11-09 05:06:49.411 RX D-Star Data
D: 2018-11-09 05:06:49.413 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.432 RX D-Star Data
D: 2018-11-09 05:06:49.433 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.451 RX D-Star Data
D: 2018-11-09 05:06:49.453 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.470 RX D-Star Data
D: 2018-11-09 05:06:49.472 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.489 RX D-Star Data
D: 2018-11-09 05:06:49.491 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.509 RX D-Star Data
D: 2018-11-09 05:06:49.510 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.529 RX D-Star Data
D: 2018-11-09 05:06:49.530 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.549 RX D-Star Data
D: 2018-11-09 05:06:49.550 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.573 RX D-Star Data
D: 2018-11-09 05:06:49.574 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.591 RX D-Star Data
D: 2018-11-09 05:06:49.593 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.610 RX D-Star Data
D: 2018-11-09 05:06:49.612 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.630 RX D-Star Data
D: 2018-11-09 05:06:49.631 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 16 29 F5 *.......BnRQ..).*
D: 2018-11-09 05:06:49.650 RX D-Star Data
D: 2018-11-09 05:06:49.652 0000: E0 11 11 0E 31 ED 3E 66 0A 13 4E 0A 55 2D 16 00 *....1.>f..N.U-..*
D: 2018-11-09 05:06:49.654 0010: 2F */*
D: 2018-11-09 05:06:49.656 D-Star, raw RSSI: 47, reported RSSI: -47 dBm
D: 2018-11-09 05:06:49.669 RX D-Star Data
D: 2018-11-09 05:06:49.670 0000: E0 0F 11 2D 6D AE 00 66 49 2D 0D 6B EC 14 C0 *...-m..fI-.k...*
D: 2018-11-09 05:06:49.689 RX D-Star Data
D: 2018-11-09 05:06:49.690 0000: E0 0F 11 4D 71 A4 88 66 49 2D 38 0A EC 00 D1 *...Mq..fI-8....*
D: 2018-11-09 05:06:49.709 RX D-Star Data
D: 2018-11-09 05:06:49.710 0000: E0 0F 11 0E 31 ED 41 66 75 6C 31 2A E4 31 ED *....1.Aful1*.1.*
D: 2018-11-09 05:06:49.729 RX D-Star Data
D: 2018-11-09 05:06:49.730 0000: E0 0F 11 71 4E 92 1B 66 31 22 72 76 E4 31 ED *...qN..f1"rv.1.*
D: 2018-11-09 05:06:49.753 RX D-Star Data
D: 2018-11-09 05:06:49.754 0000: E0 0F 11 2D 92 AE 00 66 49 2D 31 2E E4 14 C0 *...-...fI-1....*
D: 2018-11-09 05:06:49.772 RX D-Star Data
D: 2018-11-09 05:06:49.774 0000: E0 0F 11 D0 0F AE 00 66 6C 26 77 1F E4 00 D1 *.......fl&w....*
D: 2018-11-09 05:06:49.792 RX D-Star Data
D: 2018-11-09 05:06:49.794 0000: E0 0F 11 0E 0C C2 03 66 25 2E 61 55 E4 0D CB *.......f%.aU...*
D: 2018-11-09 05:06:49.812 RX D-Star Data
D: 2018-11-09 05:06:49.813 0000: E0 0F 11 0E 37 49 73 66 80 A2 FB E2 E4 07 ED *....7Isf.......*
D: 2018-11-09 05:06:49.832 RX D-Star Data
D: 2018-11-09 05:06:49.833 0000: E0 0F 11 00 9E 9E 32 66 78 64 E6 2F E4 83 D8 *......2fxd./...*
D: 2018-11-09 05:06:49.851 RX D-Star Data
D: 2018-11-09 05:06:49.853 0000: E0 0F 11 4D 0F B9 CC 66 70 DE 6B 6E E4 1A 43 *...M...fp.kn..C*
D: 2018-11-09 05:06:49.870 RX D-Star Data
D: 2018-11-09 05:06:49.871 0000: E0 0F 11 16 29 F5 26 66 12 0B 56 4D F3 00 D1 *....).&f..VM...*
D: 2018-11-09 05:06:49.889 RX D-Star Data
D: 2018-11-09 05:06:49.890 0000: E0 0F 11 16 29 F5 26 66 12 0B 56 4D F3 12 F5 *....).&f..VM...*
D: 2018-11-09 05:06:49.909 RX D-Star Data
D: 2018-11-09 05:06:49.910 0000: E0 0F 11 EE C2 A1 C8 42 6E 52 51 C3 55 55 55 *.......BnRQ.UUU*
M: 2018-11-09 05:06:49.916 Debug: DStarRX: Found end sync in Data
D: 2018-11-09 05:06:49.923 RX D-Star EOT
D: 2018-11-09 05:06:49.925 0000: E0 03 13 *...*
M: 2018-11-09 05:06:49.926 D-Star, received RF end of transmission, 0.7 seconds, BER: 0.0%, RSSI: -47/-47/-47 dBm
So, my theory is that there is something special about EE C2 A1 C8 42 6E 52 51 C3, such that it won't be generated by AMBE+FEC, or perhaps it's just extremely unlikely to occur 21 times in a row. Further, I imagine that 9-byte sequence itself validates against FEC (just guessing, I don't yet know how to check this).
Anyway, if my theory is correct, we should be able to support fast data through hotspots (and repeaters?) by turning off FEC recalculation in the case that we see 21 of these in a row.
I wonder if there's a "re-enable voice" sequence that indicates we should re-enable FEC, supporting Icom's apparent ability to move between the two modes in a single transmission. (I say "apparent" because I think this is possible based on the Icom video that demos text and image sending while also being able to send voice during the transfer). Maybe that's what 16 29 F5 26 66 12 0B 56 4D is for?
I should note that this dump was created by MMDVMHost with the FEC recalculation disabled, which is why you see a 0% BER.
Also, I said above that "I imagine that 9-byte sequence itself validates against FEC." If memory serves, these bytes contributed to the 12.6% BER in my original tests, so I think my statement is wrong. I'll double-check and confirm, though I don't think it affects my theory much overall.
That is very very interesting. What you say makes sense. I think I can move forwards with what you have told me. If you could investigate the Icom mode switch, and also if Icom use the same system, I would appreciate it. I have some ideas and I'll try and implement them next week. Unfortunately the gateway will also need modifying which is not ideal, but necessary.
Hi, I have done some tests with the same file (sent.log) and my ID51 with Fast DV data. Unfortunately it looks different from Kenwood format:
D: 2018-11-11 23:54:54.411 RX D-Star Header
D: 2018-11-11 23:54:54.411 0000: E0 2E 10 40 00 00 43 41 36 4A 41 55 20 47 43 41 *[email protected] GCA*
D: 2018-11-11 23:54:54.411 0010: 36 4A 41 55 20 43 20 20 20 20 20 20 20 45 43 41 *6JAU C ECA*
D: 2018-11-11 23:54:54.411 0020: 36 4A 41 55 20 20 20 20 20 20 56 C0 00 04 *6JAU V...*
M: 2018-11-11 23:54:54.411 D-Star, received RF header from CA6JAU / to E
D: 2018-11-11 23:54:54.430 RX D-Star Data
D: 2018-11-11 23:54:54.430 0000: E0 11 11 B2 4D 22 48 C0 16 28 26 C8 55 2D 16 00 *....M"H..(&.U-..*
D: 2018-11-11 23:54:54.430 0010: 04 *.*
D: 2018-11-11 23:54:54.449 RX D-Star Data
D: 2018-11-11 23:54:54.449 0000: E0 0F 11 AE CC 2A 78 E1 13 3C 67 C0 30 0E FD *.....*x..<g.0..*
D: 2018-11-11 23:54:54.468 RX D-Star Data
D: 2018-11-11 23:54:54.468 0000: E0 0F 11 CE 8E 2E 39 66 12 78 31 B0 14 36 B3 *......9f.x1..6.*
D: 2018-11-11 23:54:54.492 RX D-Star Data
D: 2018-11-11 23:54:54.492 0000: E0 0F 11 AE CC 2A 78 E1 91 34 67 C0 31 1A E1 *.....*x..4g.1..*
D: 2018-11-11 23:54:54.511 RX D-Star Data
D: 2018-11-11 23:54:54.511 0000: E0 0F 11 CA 8E 26 19 26 1E 4C A1 F0 19 2D F6 *.....&.&.L...-.*
D: 2018-11-11 23:54:54.529 RX D-Star Data
D: 2018-11-11 23:54:54.529 0000: E0 0F 11 AE CC 2A 78 E1 13 3C 67 C0 32 6F BC *.....*x..<g.2o.*
D: 2018-11-11 23:54:54.547 RX D-Star Data
D: 2018-11-11 23:54:54.547 0000: E0 0F 11 BE 6E 22 48 23 1B 0C 77 80 50 06 D7 *....n"H#..w.P..*
D: 2018-11-11 23:54:54.572 RX D-Star Data
D: 2018-11-11 23:54:54.572 0000: E0 0F 11 CE 8E 22 09 E4 1E 4C 20 F4 33 7A A2 *....."...L .3z.*
D: 2018-11-11 23:54:54.590 RX D-Star Data
D: 2018-11-11 23:54:54.590 0000: E0 0F 11 AA CC 2A 78 E1 91 3C 67 C0 35 6F B3 *.....*x..<g.5o.*
D: 2018-11-11 23:54:54.612 RX D-Star Data
D: 2018-11-11 23:54:54.612 0000: E0 0F 11 CE CE AC 31 47 1E 58 E0 F0 16 29 F5 *......1G.X...).*
D: 2018-11-11 23:54:54.629 RX D-Star Data
D: 2018-11-11 23:54:54.629 0000: E0 0F 11 AE 8C 2E 68 22 1F 1C 37 84 16 29 F5 *......h"..7..).*
D: 2018-11-11 23:54:54.651 RX D-Star Data
D: 2018-11-11 23:54:54.651 0000: E0 0F 11 50 26 E0 60 66 15 4D 44 4E E4 1B FB *...P&.`f.MDN...*
D: 2018-11-11 23:54:54.668 RX D-Star Data
D: 2018-11-11 23:54:54.668 0000: E0 0F 11 03 3B B3 2F 66 12 4D 44 43 E4 26 E0 *....;./f.MDC.&.*
D: 2018-11-11 23:54:54.691 RX D-Star Data
D: 2018-11-11 23:54:54.691 0000: E0 0F 11 15 3D F4 25 66 1A 0E 49 0B E4 2A B3 *....=.%f..I..*.*
D: 2018-11-11 23:54:54.708 RX D-Star Data
D: 2018-11-11 23:54:54.708 0000: E0 0F 11 12 3D FC 21 66 10 0E 51 58 E4 2A FE *....=.!f..QX.*.*
D: 2018-11-11 23:54:54.732 RX D-Star Data
D: 2018-11-11 23:54:54.732 0000: E0 0F 11 03 3B F6 2D 66 5A 4D 10 7F E4 3B B3 *....;.-fZM...;.*
D: 2018-11-11 23:54:54.751 RX D-Star Data
D: 2018-11-11 23:54:54.751 0000: E0 0F 11 18 26 E0 60 66 1D 1E 10 44 E4 3C EA *....&.`f...D.<.*
D: 2018-11-11 23:54:54.768 RX D-Star Data
D: 2018-11-11 23:54:54.768 0000: E0 0F 11 11 6F E7 25 66 07 19 1E 26 FD 21 FF *....o.%f...&.!.*
D: 2018-11-11 23:54:54.793 RX D-Star Data
D: 2018-11-11 23:54:54.793 0000: E0 0F 11 7A 4F 93 40 66 74 6D 30 2B FD 36 B3 *...zO.@ftm0+.6.*
D: 2018-11-11 23:54:54.811 RX D-Star Data
D: 2018-11-11 23:54:54.811 0000: E0 0F 11 CE 8E A0 01 C4 1E 4C 20 F4 16 29 F5 *.........L ..).*
D: 2018-11-11 23:54:54.829 RX D-Star Data
D: 2018-11-11 23:54:54.829 0000: E0 0F 11 AA CC 2E 68 23 91 3C E6 C4 16 29 F5 *......h#.<...).*
D: 2018-11-11 23:54:54.852 RX D-Star Data
D: 2018-11-11 23:54:54.852 0000: E0 0F 11 CE 8E 2E 39 66 12 78 31 B0 55 55 55 *......9f.x1.UUU*
D: 2018-11-11 23:54:54.852 RX D-Star EOT
D: 2018-11-11 23:54:54.852 0000: E0 03 13 *...*
M: 2018-11-11 23:54:54.858 D-Star, received RF end of transmission, 0.5 seconds, BER: 0.0%
I've realized that not always starts with the same data sequence, after repeating the same transmission with a ID51. For example, if I transmit a single "A" (I "cleaned" the log for better understanding):
70 4F 93 40 64 74 6D 30 2B 55 2D 16 *...pO.@dtm0+U-.*
70 4F 93 40 64 74 6D 30 2B F2 0E 99 *...pO.@dtm0+...*
70 4F 93 40 64 74 6D 30 2B F2 4F 93 *...pO.@dtm0+.O.*
CE 8E 2E 39 66 12 78 31 B0 30 0E FD *......9f.x1.0..*
AE CC 2A 78 E1 91 34 67 C0 14 36 B3 *.....*x..4g..6.*
CA 8E 26 19 26 1E 4C A1 F0 31 1A E1 *.....&.&.L..1..*
AE CC 2A 78 E1 13 3C 67 C0 19 2D F6 *.....*x..<g..-.*
BE 6E 22 48 23 1B 0C 77 80 32 6F BC *....n"H#..w.2o.*
CE 8E 22 09 E4 1E 4C 20 F4 50 06 D7 *....."...L .P..*
AA CC 2A 78 E1 91 3C 67 C0 33 7A A2 *.....*x..<g.3z.*
CE CE AC 31 47 1E 58 E0 F0 35 6F B3 *......1G.X..5o.*
AE 8C 2E 68 22 1F 1C 37 84 16 29 F5 *......h"..7..).*
CE CE 2A 29 A5 1E 58 61 F4 16 29 F5 *.....*)..Xa..).*
AA CC 2A 78 E1 13 3C 67 C0 55 55 55 *.....*x..<g.UUU*
And if I repeat the same "A":
B2 4D 22 48 C0 16 28 26 C8 55 2D 16 *....M"H..(&.U-.*
AE CC 2A 78 E1 13 3C 67 C0 30 0E FD *.....*x..<g.0..*
CE 8E 2E 39 66 12 78 31 B0 14 36 B3 *......9f.x1..6.*
AE CC 2A 78 E1 91 34 67 C0 31 1A E1 *.....*x..4g.1..*
CA 8E 26 19 26 1E 4C A1 F0 19 2D F6 *.....&.&.L...-.*
AE CC 2A 78 E1 13 3C 67 C0 32 6F BC *.....*x..<g.2o.*
BE 6E 22 48 23 1B 0C 77 80 50 06 D7 *....n"H#..w.P..*
CE 8E 22 09 E4 1E 4C 20 F4 33 7A A2 *....."...L .3z.*
AA CC 2A 78 E1 91 3C 67 C0 35 6F B3 *.....*x..<g.5o.*
CE CE AC 31 47 1E 58 E0 F0 16 29 F5 *......1G.X...).*
AE 8C 2E 68 22 1F 1C 37 84 16 29 F5 *......h"..7..).*
70 4F 93 40 64 74 6D 30 2B F2 0E 99 *...pO.@dtm0+...*
70 4F 93 40 64 74 6D 30 2B F2 4F 93 *...pO.@dtm0+.O.*
CE CE 26 19 27 12 6C 70 B0 16 29 F5 *.....&.'.lp..).*
AE EC 2A 78 E1 13 3C 67 C0 16 29 F5 *.....*x..<g..).*
CE CE 2E 39 67 1E 58 E0 F0 55 55 55 *......9g.X..UUU*
I did more experiments, trying to decode the frames. Now I transmitted: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
I got:
My guess: A: a kind of data start?. It seems to me always the same, but for some transmissions is not there! B: sync flag C: data header?, it looks always the same and it is always there. It doesn't change if I modify D-Star fields, for example. D: Slow data, here has my TX message "Andy Uribe / ID51E " E: data payload
The data payload seems to be scrambled with: 70 4F 93 40 64 74 6D 30 2B E2 4F 93 (not sure about 64 and E2). If I de-scramble the payload:
45 46 47 48 02 49 4A 4B 4C 06 41 42 *EFGH.IJKL.AB*
4D 4E 4F 50 02 51 52 53 54 06 43 44 *MNOP.QRST.CD*
59 5A 61 62 02 63 64 65 66 06 55 56 *YZab.cdef.UV*
67 68 69 6A 02 6B 6C 6D 6E 06 57 58 *ghij.klmn.WX*
73 74 75 76 02 77 78 79 7A 06 6F 70 *stuv.wxyz.op*
30 31 32 33 02 34 35 36 37 06 71 72 *0123.4567.qr*
00 00 00 00 02 00 00 00 00 10 38 39 *..........89*
00 00 00 00 02 00 00 00 00 10 00 00 *............*
It looks like the Kenwood data is the easiest to support. I think the data marker is transmitted multiple times to make sure that you receive it. You'll find that 21 frames occurs a lot in D-Star and is the distance between sync frames, so it sort of makes sense to use that as the basic unit of data transmissions. Once I have reliably detected the data mode, I'd just switch off the FEC code and let it continue. I'd probably also regenerate the data marker also for good measure. I'm still looking at the Icom version. In theory switching off the FEC will be fine if the system switches back to voice, you just won't get the benefit of the FEC for the rest of that transmission. I do wish that Kenwood and Icom had used a bit in the header instead or as well as, to make my life easier.
I am going to repeat the same tests with a ID-4100A and see if I get the same results. Something that I forget to mention: after all my tests with ID-51A, the first frame could be one of the following two possibilities:
- The sequence: B2 4D 22 48 C0 16 28 26 C8
- Directly an "almost" empty payload: 70 4F 93 40 64 74 6D 30 2B, where the following slow data will contain the first 4 bytes of the message.
Anyway, it seems to me that the large structure "C" is always in any Icom fast DV transmission.
Unfortunately, I don't have any Icom hardware to test with :( I'm hoping to get an Icom ID-4100A by the end of the year, though.
I have received an initial response from my email to Kenwood:
- The TH-D74A supports fast DV data
- The TH-D74A does not support slow DV data (to/from a connected computer; obviously it's using slow data for the display messages)
- The TH-D74A's fast data is compatible with Icom's fast data
The last point could mean the D74A implements a simpler subset, I suppose, since it doesn't support switching back and forth between fast and slow.
I haven't gotten a negative response from either Icom or Kenwood support about getting tech specs, so I'll update here when I have any answers there.
Maybe there is some shared property of the bytes I saw (EE C2 A1 C8 42 6E 52 51 C3) and block C of @juribeparada's dump, e.g. some checksum they both validate against, and Kenwood just implemented something simpler. But I'm just shooting in the dark here :)
Anyway, I plan to poke around a bit more with the TH-D7A this week and compare to the other posts here, in case I can discover anything.
Thanks for looking at this!
I've added some code to the D-Star support in the host. It should detect the use of fast data mode, and log it. It should work on RF and network although the ircDDB Gateway needs to be modified before we can send fast data over any network. For now, if you can test/modify the code that I've added for the fast data mode.
I had little time for testing today, but I could check:
- ID-4100A sends the same frames as ID-51 in fast DV mode, with the two "identifiers".
- The current MMDVMHost detects both Icom identifiers well, and works well with ID-51 and ID-4100A in fast DV data.
I've added data marker regeneration on RF and network traffic. I've checked the ircDDB Gateway and it does no AMBE FEC regeneration so fast data should pass through it without problems. The only potential issue is the DTMF decoder which could trigger and blank the data. This will need testing to make sure what goes in, comes out unchanged. I'm hoping that we can get this finished soon.
BTW in your data, I wonder if element C is your RF header, scrambled. The only way to be sure would be for someone else to generate some Icom fast data with a dissimilar set of callsigns and do a comparison.
It seems to me the block "C" doesn't change: I used different D-Star fields YOUR, RPT1, etc (except the callsign, I've used always the same), even I transmitted accidentally in simplex with the 4100, and I captured the same "C" structure, something curious. I will be out of the town for 10 days, then I will be not able to do more tests until come back.
The changes look good to me so far. I was able to send and receive the same text using an echo test, and I get the Kenwood message in the logs.
Now I'm hanging out on REF030D in D-RATS if either of you are around and want to test.
I've shared a file via D-RATS in case one of you decide to test via REF030D while I am away. I realize this may not work yet, but I thought it might be fun to try.
Goodnight!
-TimS, KG4BXH
That's very pleasing. I am still worried about the DTMF processing in the gateway which could potentially corrupt the data if triggered by random data. I will look to switch it off in data mode. I also need to think about the older D-Star Repeater software, whether to upgrade it or not. It will be interesting if there is a switch in the Icom data mode to go back to audio, as things stand it should work anyway, but any following audio won't have the extra processing that audio gets now. Sorry I wasn't on D-RATS but I live in the UK and the time difference killed that, as well as other things.
I suspected you were in the UK and it was a long shot :) Also, my pi-star decided to restart all services at 3:30am EST anyway, so that killed the test after a while... need to turn that off. So, to be clear, my D-RATS test is now over, heh.
Anyway, I'm still waiting on potential docs from Kenwood and Icom. That would be really nice if one of them could just tell us how it works. I'll see if I can find someone willing to test with me, given that I have only one radio.
I think I could help out with testing. I have a ID51 at least but neeed to figure out how fast data works.
I'm in the same boat Flo. I have a TH-D74 and an ID-51 and haven't used fast data on either of them. That's why it's never been added to my work before. I've updated the ircDDB Gateway to bypass all DTMF handling when fast data is detected, but it's on a branch because I can't build it currently. It'll be rolled into master when it's been compiled cleanly.
In my case, I enabled Fast DV data from ID-51 menu (DV Set->DV Fast Data->Fast Data->ON), and then simply I send the data using the serial port of the radio at 38400 bps:
echo "Test DV fast data" > /dev/ttyXXX cat file.txt > /dev/ttyXXX
If fast data is OFF, all data will be sent as slow data as usual.
I finally have my fast data equipment up and running. Sorry for the delay. Anyone up for a test (across reflectors)?
@phl0 @ravny If anyone is around this weekend, I have D-RATS listening again on REF030D, and I'm serving up a text file if you browse my files.