libusb
Interrupts from individual isochronous packets
I am implementing a custom FS USB device which sends a 50 to 300 byte isochronous IN transfer every 10 to 25ms to a host PC, with minimal latency.
Currently, I'm at the point where the device sends a small isochronous packet to the host PC every 15th frame. Now, I want to receive this packet in the host application with minimal latency. However, I haven't managed to get the isochronous transfers to work with a package count smaller than 8, which means the transfer waits for 8 whole frames and only then I can see the data that was sent, which is suboptimal.
Say I have the following values:
#define ISO_PACKET_SIZE 128
#define NUM_PACKETS_BUF 8
#define NUM_PACKETS_ALLOC 8
#define NUM_PACKETS_FILL 8
And I set up two isochronous IN transfers isoIN1 and isoIN2 to always have an active transfer:
alignas(2) uint8_t isoINxBuf[ISO_PACKET_SIZE*NUM_PACKETS_BUF];
----
isoINx = libusb_alloc_transfer(NUM_PACKETS_ALLOC);
libusb_fill_iso_transfer(isoINx, devHandle, 2 | LIBUSB_ENDPOINT_IN, isoINxBuf, sizeof(isoINxBuf), NUM_PACKETS_FILL, &onIsochronousIN, state, 1000);
libusb_set_iso_packet_lengths(isoINx, ISO_PACKET_SIZE);
Here are some logs for the values (BUF, ALLOC, FILL), with the device already connected.
libusb_DEBUG_8_8_8.txt
This configuration works, but since the transfer consists of 8 packets I only get a callback once all eight transferred. Also, this does show the following error occasionally, which does not seem to affect the behaviour though:
libusb: error [windows_transfer_callback] detected I/O error 87: [87] The parameter is incorrect.
libusb_DEBUG_8_8_1.txt libusb_DEBUG_8_1_1.txt Both act similar, the transfers fail, same error as above, although the initial submit is successful.
libusb_DEBUG_1_1_1.txt
With the buffer being a different size now, WinUSB complains about the buffer size:
libusb: debug [winusbx_submit_iso_transfer] The length of isochronous buffer must be a multiple of the MaximumBytesPerInterval * 8 / Interval
This might just be the root cause, however I have not found any reason for limiting isochronous transfers to a minimum of 8 packets.
In case it's faulty, here's the configuration descriptor: libusb_DEBUG_Configuration.txt
Any ideas how I can get around this? In the end all I care about is a timely callback for when a non-zero-length isochronous packet got received. I do expect most isochronous transfers to be zero-length, so if I could save some precious resources not handling those, even better (software is designed to run along highly demanding software out of my scope). Thanks for any input.
Isoc is not quite suited to your need , interrupt type end point could be better as you data rate nor poll rate is excessive. Isoch packet sorting is a bit more complex to handle to manage. Still this does not help you much :) I'm not 100%sure isoch and wnusb is fully functional i mainly used bulk/interrupt. could be worth trying another back-end (libusbk)
Ok thanks! I do have interrupt working on the alt_0 interface, however I'm currently stuck with a FS USB device (STMF103) with a maximum interrupt size of 64, which is not enough. If I can I would like to use it in the final version as well for other reasons.
However, I think I found part of the issue. In the WinUSB Isochronous transfer implementation, I believe it makes the assumption that we're using HighSpeed (it assumes microframes): https://github.com/libusb/libusb/blob/master/libusb/os/windows_winusb.c#L2642 I will try changing this for Fullspeed devices and see if WinUSB complains. If not, then (1,1,1) - only one isochronous transfer - should be work just fine. This would also explain why (8,1,1) or (8,8,1) didn't work (and why they are equivalent), WinUSB only sees the 1 transfer but a buffer for 8 transfers and that's invalid.
If that does not work, I will try changing this in hopes it will get me a callback on every received isochronous transfer: https://github.com/libusb/libusb/blob/master/libusb/os/windows_winusb.c#L2515 Although if this really worked, it would not be something we could easily merge into libusb, since others will expect the callback when all transfers completed successfully.
Found one issue in the current repository state,
This line passes SUB_API_NOTSET to the copy_transfer_data implementation, and this seems to have been the case for at least the past 5 years.
That subapi is usually set in the function itself, using the macro CHECK_WINUSBX_AVAILABLE.
However, this macro was removed in a recent commit, causing the subapi in this function to always be SUB_API_NOTSET.
This of course triggers the else case and results in an error:
libusb: debug [winusbx_copy_transfer_data] unsupported API call for 'copy_transfer_data' (unrecognized device driver)
Possible solutions:
- Adding it back in along with
struct winusb_device_priv *priv = usbi_get_device_priv(transfer->dev_handle->dev); - Use the function as it was designed and pass the
priv->sub_apito the function
Now that libusb is working as before, I'll investigate the actual matter.
So it turned out changing this line as I intended did the job and most everything worked as expected.
However the occasional error even in the (8,8,8) setup remained and had a deeper cause, that being this behaviour. The ContinueStream parameter of the WinUSB API provides a way to enforce the next transaction to be scheduled for the frame after the last one queued. If it can't, it throws an error. If used properly this is a good way to make sure you didn't loose a packet and be notified if you did. While this is nice and all, libusb provides no API to set this, so the implementation makes some guesswork which can create problems. It first tries if it can continue the stream with ContinueStream=True, if it can't (e.g. when there was no stream before to continue), it tries again with ContinueStream=False.
If you have only one transfer, this obviously fails every frame due to the delay of the resubmission. However, the general behaviour libusb recommends to address this resubmission delay is to have two transfers, so that at least one is active at each time. However when you follow this advice with isochronous IN transfers, the transfers will still fail to establish a stream, since all submitted transfers will fail simulatenously, then each trying to start a new stream. However the following transfers with ContinueStream=False can not attach after the first one to continue the stream, as WinUSB seems to force a delay if ContinueStream is false. I can verify that it is exactly 5 frames.
But with perfect timing you could get it to work, if the other transfers are submitted after the first one is submitted with ContinueStream=false and before it finishes. Submitting the second transfer immediately when the first one finishes in hopes of it continuing the stream also does not work, because of the delay of callbacks. The most practical solution would be an additional parameter to explicitly set ContinueStream, however, since this would add to the API, and ONLY for the WinUSB backend at that, this is probably not the way to go for libusb.
So alternatively, I changed the current behaviour so that the user callback is fired with a new event (e.g. Stalled, which is not used for isochronous transfers otherwise) that notifies the user that a transfer with ContinueStream=True was rejected because it could not continue the stream:
- An ongoing stream was interrupted, but libusb will attempt to continue it (loosing at least 5 frames)
- The there does not exist a stream to continue and libusb will continue to establish a new one
This for once brings back the reason ContinueStream exists in the first place, to detect if a stream was interrupted. Also, with some additional signaling capabilities this setup allows for great control for the application to handle stream interrupts/stalls. I used the hacky solution of modifying transfer->callback in the user allback as signaling, as it has been used by the existing solution before either way. When the user callback overwrites the transfer->callback with a new callback after a stall event, it will be called after the current transfer is resubmitted with ContinueStream=False. This allows the user to submit new transfers right after the previous one is resubmitted to start the stream, which allows the user to continue the stream with new transfers. This is only relevant for starting the stream, after it is started, it more or less self-regulates.
Just that gave pretty good results already. On a load-free system I no more had any stalls (after initial 100ms stall where transfers were submitted), with 16 simultaneous transfers. Even with small load spikes (opening a program) there were no stalls, and if there were, they recovered quickly, usually within 6-10ms. Now much more interesting is a system on load, I simulated a load of 30-40% on all cores (youtube 720p video stream, 720p recording that stream). This was still very stable, with quick recovery on load spikes. Then with a simulated load of 60-100% (avg 75%) on all 6 cores (youtube 720p video stream, devtools with network monitoring open, 720p recording that stream), it is less stable and barely usable. After an increase to 32 simultaneous transfers, it is better and very much usable, with callback latency below 2ms, with unavoidable losses only on some load spikes.
I guess with a but smarter userspace management you could get quicker recovery and less packet loss, but I'll have to try out. It's not optimal but it works for now and is probably lower-latency than sending multiple 64-bytes interrupts for my use case. The changes to libusb are very few, only a few lines in three places, and only affect the winusb backend so I would try to create a PR when I'm happy with it. If you have any ideas how to handle signaling better (Alternatives to STALL event in isochronous callback or using transfer->callback for signaling). Currently, it COULD break applications which do NOT ignore the Stall event in the isochronous callback.
Alright, created a PR. Here's some example code that I quickly adapted from my code. Makes use of these changes to create a pretty solid isochronous stream.
#define USE_WINUSB_BACKEND
#define DEBUG_STALL_RECOVERY
#define ISO_PACKET_SIZE 128
#define NUM_PACKETS 1
#define NUM_TRANSFERS 32
alignas(2) uint8_t isoINBuf[NUM_TRANSFERS][ISO_PACKET_SIZE*NUM_PACKETS];
libusb_transfer *isoIN[NUM_TRANSFERS];
#ifdef USE_WINUSB_BACKEND
std::atomic<libusb_transfer*> g_isoINStalled;
std::atomic<bool> g_isoINSubmittedALL;
#endif
#ifdef DEBUG_STALL_RECOVERY
std::chrono::time_point<std::chrono::steady_clock> g_lastStallStart;
std::atomic<bool> g_stalling;
static StatValue g_stallRecovery;
std::atomic<int> g_stallTransferCount;
#endif
/** On Init */
for (int i = 0; i < NUM_TRANSFERS; i++)
{
isoIN[i] = libusb_alloc_transfer(NUM_PACKETS);
libusb_fill_iso_transfer(isoIN[i], devHandle, 2 | LIBUSB_ENDPOINT_IN, isoINBuf[i], sizeof(isoINBuf[i]), NUM_PACKETS, &onIsochronousIN, state, NUM_TRANSFERS*10);
libusb_set_iso_packet_lengths(isoIN[i], ISO_PACKET_SIZE);
}
#ifdef _WIN32 && USE_WINUSB_BACKEND
// Submit only first transfer to start the stream, and wait for the first stall, after which to submit the rest
code = libusb_submit_transfer(isoIN[0]);
printf("Submit Isochronous transfer 1 (%p): %s!\n", isoIN[0], libusb_error_name(code));
g_isoINSubmittedALL = false;
g_isoINStalled = NULL;
#else
// Submit all at once normally
int code;
for (int i = 0; i < NUM_TRANSFERS; i++)
{
code = libusb_submit_transfer(isoIN[i]);
printf("Submit Isochronous transfer %d (%p): %s!\n", i+1, isoIN[i], libusb_error_name(code));
}
#endif
/** Callbacks */
#ifdef _WIN32 && USE_WINUSB_BACKEND
static void onIsoINSubmission(libusb_transfer *transfer)
{
g_isoINSubmittedALL = true;
int code;
for (int i = 1; i < NUM_TRANSFERS; i++)
{
code = libusb_submit_transfer(isoIN[i]);
printf("Submit Isochronous transfer %d (%p): %s!\n", i+1, isoIN[i], libusb_error_name(code));
}
}
#endif
static void onIsochronousIN(libusb_transfer *transfer)
{
switch (transfer->status)
{
case LIBUSB_TRANSFER_COMPLETED: break;
#ifdef _WIN32 && USE_WINUSB_BACKEND
case LIBUSB_TRANSFER_STALL:
// Isochronous stream got interrupted OR just started (only relevant for libusb WinUSB backend)
// For the case that this is the first transfer and the stream was not started yet:
// The current transfer is isoIN1, and it will get resubmitted by libusb AFTER this callback
// We have the chance to submit all our other transfers right AFTER the isoIN1 is get resubmitted
// By overwriting the transfer callback temporarily
if (g_isoINStalled == NULL)
{ // Have not handled stall before, resubmit this transfer to initiate new stream
#ifdef DEBUG_STALL_RECOVERY
printf("(%p) reported stream interruption!\n", transfer);
g_lastStallStart = std::chrono::high_resolution_clock::now();
g_stallTransferCount = 0;
#endif
g_isoINStalled = transfer;
if (!g_isoINSubmittedALL)
{ // Some transfers are not (yet) submitted, do so after this one has been resubmitted with ContinueStream=False
transfer->callback = onIsoINSubmission;
}
}
// Setting transfer->callback = NULL will supress automatic resubmission with ContinueStream=False alltogether
#ifdef DEBUG_STALL_RECOVERY
g_stalling = true;
g_stallTransferCount++;
#endif
return;
#endif
default:
// TODO: Handle other events as usual
return;
}
if (g_isoINStalled == transfer)
{ // Successfully unstalled previous stall
g_isoINStalled = NULL;
}
#ifdef DEBUG_STALL_RECOVERY
if (g_stalling)
{ // Transfer completed successfully after stalling -> recovery
g_stalling = false;
auto now = std::chrono::high_resolution_clock::now();
float recoveryTime = std::chrono::duration_cast<std::chrono::microseconds>(now - g_lastStallStart).count() / 1000.0;
printf("(%p) recovered from stall affecting %d transfers after %.2fms\n", transfer, g_stallTransferCount.load(), recoveryTime);
}
#endif
// TODO: Read out packets and such
libusb_submit_transfer(transfer);
}
@Seneral Just wondering if you have a full example test program. I am not good at coding and I do not have a good C code to carry out isoc related test. Thanks.
Ref:
- https://github.com/libusb/libusb/issues/1104