[Enhancement] Non-streaming transmit and receive in libhackrf?

[CFSworks]

Hi!

This issue is mainly testing the waters for interest in an enhancement to add "non-streaming" transmit/receive capability to libhackrf. This would add a hackrf_tx_now/hackrf_rx_now or similar, which takes a fixed-length buffer and enters TX/RX mode only long enough to send/fill (respectively) that buffer. This would support the following use cases:

1. Some applications (such as hackrf_transfer) do not multitask and as such implementing a pair of streaming callbacks is unnecessary control flow. A beginner who only cares about writing one main function in their program can use HackRF synchronously.
2. Integrating HackRF support into a larger, non-thread-safe program (e.g. as a plugin) becomes easier, since the full buffer can be rendered in the plugin code and transmitted at once (synchronously or asynchronously).
3. Most importantly, if these functions can be made to work even while libhackrf is already in streaming mode, they can be used to momentarily switch transceiver modes for half-duplex operation. This allows HackRF users to implement bidirectional protocols that require acknowledgements: the HackRF can remain in receive-streaming mode, and whenever a packet comes in that needs acknowledgement, the non-streaming transmit function is used to send the acknowledgement. The receive stream is padded with zeros for the duration the HackRF is in transmit mode so the receiver doesn't lose timing.

A few considerations:

1. This will most likely need special attention in the firmware. We will need to add a new hackrf_vendor_request at least to support this operation, and perhaps even add a second pair of bulk endpoints to transfer the non-streaming request while the streaming operations continue.
2. The HackRF does not switch TX/RX modes instantly; I seem to recall someone saying it took about 100us. Should the function return timing information about how many streaming samples were dropped to make way for the non-streaming buffer?
3. Should these functions be synchronous? Asynchronous returning a handle that can be checked/waited on? Asynchronous with a completion callback? What arguments should they take?
4. If asynchronous, what should happen if someone calls the function twice in a row (i.e. before the previous operation finishes)?
5. Should these functions support overriding transmit streaming to transmit a different finite buffer, and vice versa for receive?

Most importantly: Is the demand for this worth the change to the fundamental operation of the library/firmware to support cutting in the middle of an existing stream?

┆Issue is synchronized with this Basecamp todo by Unito

[dominicgs]

Sam, thanks for this suggestion, especially the long form way that it's presented, it helps me to understand it and I really appreciate it. I'm sorry it's taken me a few days to get back to you.

This probably ties in nicely with an recurring request that I'm trying to address (#322), which is robust Tx/Rx switching. I think the most common use case for this is transmitting a fixed buffer and falling back to receive, but I think it's worthwhile implementing it symmetrically.

I'm reworking the transceiver code to allow for callbacks and to separate out the state of USB and RF sections. This should help me to fix an issue we've seen using similar firmware code on GreatFET, which is that we couldn't use the IN and OUT USB endpoints simultaneously.

If the two endpoints work simultaneously, then the solution for transmitting a buffer while receiving is simple: Present data to be transmitted to the OUT buffer HackRF receives the data, switches to TX mode Transmits buffer Once buffer is transmitted, switch back to RX mode and pick up where we left off

One downfall of this is that we'd need to add some metadata for setting frequency, gain, etc

I'm not sure of how this would work in reverse, probably a vendor request like the one that you suggest "receive X bytes now".

[CFSworks]

Hey Dominic!

Glad I was able to provide a new perspective on #322! And yeah, my main motivation for this enhancement proposal is better TX/RX switching too - the other use cases are really just benefits of it not being a callback.

I hadn't considered the possibility of pushing data into the TX bulk endpoint and having HackRF itself switch in response to data being available. It certainly keeps the USB communication very straightforward, but my two main concerns with that are:

1. There's no solid signaling mechanism for the host to tell the HackRF that it's done filling the buffer - if the HackRF only transmits when there's data in the TX buffer, it may switch too early, transmit a fragment, run out of samples, switch back to receive, then back to transmit, send another fragment, ...
2. For implementing e.g. TDMA protocols, it'd be extremely useful if the API provided a way to specify the exact sample where transmission should begin. "Sample 1,234,000 is the first sample in the guard interval and you should transmit starting at 1,300,000." This tells the HackRF when it may stop receiving (after 1,234,000 it begins switching to transmit), when it must begin transmitting (the first sample in the application-provided buffer must correlate with 1,300,000).

I'm wondering if a good way to implement all of this, firmware-wise, would be a "scheduled reconfiguration" vendor request: One can tell the HackRF what it needs to do in advance, like "set the gain to K dB at sample N" (and implement things like per-station AGC) or "change to frequency X at sample A, frequency Y at sample B, frequency Z at sample C" (and now the HackRF itself is taking care of your frequency-hopping for you - as long as you keep its scheduled-reconfigurations queue topped up, that is).

And then for the TDMA use case, as soon as the software decodes a message from the timing master saying when its time slice is, it just tells the HackRF "At sample 1,234,000, switch the RF pipeline into transmit mode. At sample 1,300,000, begin flushing the OUT buffer into the DAC. At sample 2,000,000, shut off the transmit pipeline and go back to receive."

Now, that's a LOT of API changes and I'm not suggesting all of them now (among other things it requires the HackRF to count samples in streaming mode so it knows what the host is talking about when it says "you need to stop receiving after sample 1,234,000" - and means the streaming callbacks need a way of reporting the current sample counter to the application) but it may be a good direction to start taking things. Plus I'm kind of jotting my thoughts down here as they come to me and so I'm starting to prattle on a bit and get away from what this actual issue is about. :)

---

Maybe a good starting place would be just to decouple the DAC/ADC FIFOs from the transmit/receive mode setting? So the user can have both the IN and OUT FIFOs running (meaning both of the libhackrf tx/rx callbacks are firing off at the same time - so they might want to limit their sample rate when doing this), but this lets them selectively flip between tx/rx mode without losing timing.

So that would mean:

1. Both IN and OUT bulk endpoints can be read/written (respectively) at any time, not just when in the correct mode.
2. HACKRF_VENDOR_REQUEST_SET_TRANSCEIVER_MODE only reconfigures the physical tx/rx pipeline (and can be used to e.g. prepare to transmit without transmitting yet) but does not start/stop the tx/rx FIFOs.
3. The actual transmitting/receiving begins when you do a HACKRF_VENDOR_REQUEST_SET_FIFO_STATE (or similar) which can be used to turn on/off the IN/OUT FIFOs (which fill/drain at the sample rate).
4. If the IN FIFO is running, but the transceiver isn't in receive mode, then the HackRF is just filling it with zeroes (which preserves timing).
5. Likewise, if the OUT FIFO is running, but the transceiver isn't in transmit mode, then the HackRF is just tossing samples into the bit bucket.
6. Non-streaming transmit works by putting the samples into the OUT FIFO, telling the HackRF to switch to TX mode, running the OUT FIFO until empty, then switching the HackRF back to its previous mode. (The IN FIFO, if running, is undisturbed so a concurrent streaming IN operation keeps its timing.)
7. Non-streaming receive works by switching to RX mode, running the IN FIFO for some number of samples, then switching the HackRF back to its previous transceiver mode. (Likewise leaving OUT FIFO undisturbed.)

We probably don't want to break compatibility, so that would mean the new vendor requests to add would be something like:

• HACKRF_VENDOR_REQUEST_SET_TRANSCEIVER_MODE_ONLY: only changes the physical hardware mode and LEDs, doesn't affect the FIFOs
• HACKRF_VENDOR_REQUEST_SET_FIFO_STATE: takes a FIFO index (0 or 1) and sets its running state (also 0 or 1)
• HACKRF_VENDOR_REQUEST_TRANSMIT: switches transceiver modes, empties the OUT FIFO, switches transceiver modes back
• HACKRF_VENDOR_REQUEST_RECEIVE: this would be your 'receive X bytes now' - where it puts X samples into the IN FIFO before switching back

You'd know the current state of the firmware better than I, and maybe you'll want to finish your refactor of the USB code first, but if all of this sounds good I'd be willing to start plugging away on a PR to implement these. (And if the above sounds good especially re. the sample counter I'd be willing to shove that into another issue.)

[willcode]

This sounds like a great feature. Only comment right now is that if RX and TX streams overlap for a short time during the switch, there will be USB drops at high sample rates. What latency requirements/wishes do people have?