Missing Transfers in usb analysis

[fvdpol]

When analyzing an usb trace I noticed that some of the larger bulk transfers are not present in the ViewSB output.

As a comparison I looked at same trace in Wireshark and noticed that the packets are present in the trace but missing:

One example (multiple exists in my trace....)

• the IN Transaction from packet #9307-9309 is shown in both Wireshark and ViewSB;
• the following packets #9311-9223 are missing in ViewSB (marked in Wireshark)
• from packet #9324 onwards (the PING/NAK sequence ViewSB stats showing the data)

Capture device is OpenVizsla, capturing the hs traffic to pcap file (USB link layer packets); SOF packets filtered out on both ViewSB and Wireshark.

The tui frontend is showing same issue (but more difficult to interpret/compare against the wireshark output):

[miek]

This sounds like it's related to #14 - those packets have probably been consumed by the decoder but, since the transfer hasn't finished, they haven't been emitted. We need a way to flush the partial transfer when the capture ends.

[fvdpol]

Indeed likely related to #14, especially considering the comment of xairy who noted similar missing (interrupt) packets.

I doubt that all these missing transfers are partial/incomplete; looking at the packet capture in wireshark I noticed that multiple transfers with different end-points seem be interleaved. My gut feel is that this confuses the ViewSB analyser/grouping logic.

I've extract a fragment from my initial capture in case someone is interested to reproduce this issue: missing-transfers.pcap.gz

[fvdpol]

Some further progress.... found that the USBPacketSpecializer is not correctly classifying the USB-2 High-Speed PING packets as Tokens (NAK limiting ping flow control); see diagram from the USB2 spec:

Expect to send a PR later today :-)

[fvdpol]

@miek, while digging through this issue I start to become aware that you are probably right on the money that this issue is related to the decoder not 'detecting' the end of the transfer.

Thinking about this, in my example I'm looking at an audio device that is effectively streaming data using a couple of bulk endpoints. So from looking at the low-level packets on the USB bus these transfers effectively will never 'complete'. On a higher protocol layer the data is likely transferred in packets that consist of 1 (or more) USB transfers, depending on the configured buffer sizes (probably configured during setup in the device initialization, but potentially even statically defined in the firmware).

When I disable the "USBTransferGrouper" decoder in decoders/grouping.py (setting INCLUDE_IN_ALL = False) I can nicely see the low-level packets that got missing so-far; but this disables most of the great decoding in ViewSB.

Since silently missing packets/transfers defeats the purpose of low-level usb decoding, I start to wonder if the grouping in transfers is always desirable. It almost feels like these transfers would need to be grouped per device/endpoint and shown as individual traces (like on a scope or logic analyzer) -- but this is a completely difference feature and would require discussion with the ViewSB product owner(s).

For now I'll continue to see if I can improve the USBTransferGrouper to group the various (NAK etc.) transactions related into the transfer, but add a way to terminate the Transfer so can actually see what is going on on the bus.

Any feedback/comments appreciated.

[fvdpol]

Quick update; I implemented logic in a private branch (https://github.com/fvdpol/ViewSB/tree/transfer-grouping) to stop collating transfers 'forever' and limit the grouping to the USB Packet Level.

This way the USBDataTransfer object contains a single full transfer, including handshake protocol (handling of NAKs and PINGs)

I do feel that this level of aggregation/grouping at USB protocol level makes most sense for a protocol analyzer. Collation of multiple transfers is something that in my view should be moved out and represented as a higher level object (the nitty-gritty of the NAKs/PINGs is also not interesting to higher level grouping). To be effective/correct this likely requires some knowledge of the higher-level protocol to determine when a transfer is complete.

[martinling]

Hi @fvdpol,

I've been working on using ViewSB with LUNA to look at some USB problems seen with HackRF. That device streams up to 40MB/s of data on a single bulk endpoint, so about 80,000 512-byte packets per second. Every now and again there is a control transfer. What I'm interested in is the relationship between the control transfers and the data stream.

I have also run into this issue, because the stream is never-ending so cannot be seen as "complete". But some higher level grouping is essential for me due to the sheer number of the packets. And in my case there is no internal structure to the stream which would allow it to be broken down further - it's just endless raw data.

What I'd like to see in the UI is for all the bulk packets in between other events to be placed in some sort of summary group - here's a made up example:

[+] [bulk traffic summary: EP0 IN: 10.4MB (21230 packets of 512B), EP1 OUT: 6.1MB (12456 packets of 512B)]
[+] [vendor request #N]
[+] [bulk traffic summary: EP3 OUT: 19.6MB (40295 packets of 512B), EP4 IN: 16.3MB (33295 packets of 512B)]
[+] [vendor request #M]
[+] [bulk traffic summary: EP2 IN: 3.6MB (7345 packets of 512B)]

and then only on expanding those groups, would you see the 512-byte packets within them.

I think the key thing we need is the ability for things to appear in the UI before they're complete, and be updated on the fly as more packets come in. That way, we get the benefits of high-level grouping as well as the ability to see things that are still in progress.

[fvdpol]

Hi @martinling; in my view both the Transfers at usb protocol level as well as the collation at higher protocol level are needed, and useful depending on your specific need.

Lumping both the low-level USB protocol and the higher level application layer together does not make much sense as your effective data packets will be surrounded by the (in my case) many IN/PING packets that get NAK'ed.

I like your example, and this is much in line with what I have in mind as a future improvement: move the higher-level transfers into their own group, and have the low-level usb transfers within. To get this right for pending/ongoing transfers (like in case the stream does not see an end) this will also require quite some GUI work to display these 'pending/in-progress' transfers -- which would solve long-standing solve #14

Another improvement idea I have is to add an aggregator for the consecutive NAK'ed packets (when there are a handful this is still manageable, but I have seen cases where there are many (1000 or so) PINGs before the receiver is ready.

[fvdpol]

... device streams up to 40MB/s of data on a single bulk endpoint, so about 80,000 512-byte packets per second. Every now and again there is a control transfer. What I'm interested in is the relationship between the control transfers and the data stream.

This raises an interesting challenge; currently the packets are shown as a linear list that has been turned into a tree structure. When you are interested in analysing events between different end-points, it will be key to be able to see these in a common time-domain -- something the linear list is not really helpful.

What you likely would like to see is an alternate view that allows to show multiple transfers on a common timeline; a bit like a logic analyser or oscilloscope would visualise the time relationship. This view would then show the time-relationship between the events on the bus; and the (existing) detailed view allows to zoom in to the packet details. Selecting something in the packet list would highlight the corresponding event in the time domain view, and vice versa... Hope below crude diagram helps illustrate my thought:

[martinling]

Yes, I agree that we want both levels of grouping - the low-level 512B USB transfers, and some higher level grouping of the overall data stream they form. The existing grouping seems to have been a shortcut that wasn't ideal for either purpose. At the protocol level it obscured the low level 512B transfers, and at the higher level view, it caused a continuous data stream to be broken into rather arbitrary chunks.

I think the long standing issue you refer to is #14 rather than #10.

The mismatch between the tree structure presented by the UI and the interleaving of endpoint traffic on the wire is indeed an issue. I think eventually there does need to be a logic analyzer / oscilloscope style view of things as well as you say. For LUNA we also have the possibility to capture other logic signals alongside the USB capture, so this would integrate with that ability.

I think there might be some things that could be done within the existing tree view though, to illustrate the overlapping timings of different operations. For instance, using connecting lines to connect different groups, and using timestamp ranges rather than simply a single timestamp per item. Perhaps it might look something like this:

0.000 - 0.001    [+] Vendor request #1
0.002 - 1.234 ┌──[+] EP 1 IN: 19.6MB (40295 transfers of 512B)
0.003 - 1.123 │┌─[+] EP 2 OUT: 3.6MB (7345 transfers of 512B)
1.125 - 1.126 ││ [+] Vendor request #2
1.127 - 2.345 ├┼─[+] EP 1 IN: 6.1MB (12456 transfers of 512B)
1.128 - 2.234 │├─[+] EP 2 OUT: 19.6MB (40295 transfers of 512B)
2.400 - 2.401 ││ [+] Vendor request #3
2.402 - 3.345 └┼─[+] EP 1 IN: 6.1MB (12456 transfers of 512B)
3.500 - 3.501  │ [+] Vendor request #4
3.678 - now    └─[+] EP 2 OUT: 16.3MB (33295 transfers of 512B)

[miek]

One thing to note is that a "transfer" means something specific in the USB spec, it is made up of one or more data packets and is completed by a short packet - so labelling each individual data transaction as a "transfer" is misleading.

[fvdpol]

One thing to note is that a "transfer" means something specific in the USB spec, it is made up of one or more data packets and is completed by a short packet - so labelling each individual data transaction as a "transfer" is misleading.

Would you mind sharing a link/reference to the USB spec where this is defined? I'm primarily relying on the USB Spec revision 2.0 (April 27, 2000), and definitely want to stay true to the specification and avoid confusion or even being misleading.

Could it be you're referring to the I/O Request Packets (IRPs), since these use a 'short package' to signal completion in variable-size data amounts? Anyway, I'm more than happy to learn and up-skill my understanding of the USB protocol.

My interpretation is that if we start from the data captured on the bus (that is, for capture devices that tap into the physical layer (USB Wire), with devices like OpenVizsla, Rhododendron, Luna etc.), and work our way up; we we'll find for our data transfers USB packets, that make up Transactions (USB Framed Data), that make up Transfers (USB Framed Data). (ie. the flows on left hand corner on figure 5-9)

[miek]

Would you mind sharing a link/reference to the USB spec where this is defined? I'm primarily relying on the USB Spec revision 2.0 (April 27, 2000), and definitely want to stay true to the specification and avoid confusion or even being misleading.

It's described for each transfer type under 5.5.3, 5.6.3, 5.7.3, and 5.8.

Could it be you're referring to the I/O Request Packets (IRPs), since these use a 'short package' to signal completion in variable-size data amounts? Anyway, I'm more than happy to learn and up-skill my understanding of the USB protocol.

In a way, because they're closely related. An IRP is only a term to describe a request for a data transfer from the host software, it doesn't actually describe anything happening on the bus and any details of it are defined by the operating system (it might correspond to calling libusb_submit_transfer for example). So, when a transfer is completed by a short packet, the request for that transfer completes too.

[fvdpol]

Thanks for the feedback. I’ve updated my code to keep the transfers grouped as-is by default; and added an optional command-line argument to fragment the transfers when a specified number of bytes has been transferred.

This offers a workable workaround until we have a proper solution to visualize the ‘in progress transfers’ (issue #14) for the scenarios where transfers are huge or endless streaming.

[martinling]

Closed in preparation for archiving the repository. We have replaced ViewSB with Packetry.

[martinling]

And note that Packetry implements a lot of what was discussed here!