USB3, Ubuntu 18.04 host: LIBUSB_ERROR_OVERFLOW on TRANSFER1 (Final1) transfer

[RyanMetcalfeNovanta]

Hello! To be really clear up front -- This is more of a question as opposed to a bug report -- I'm currently implementing a device-side stack for U3V, and testing it using aravis, and I'd like to get your thoughts on some behavior I'm seeing , and whether you think it is expected / intended.

As the title indicates, when testing on an Ubuntu 18.04 host, I am receiving LIBUSB_ERROR_OVERFLOW errors (visible when setting ARV_DEBUG=all:3) during stream transfers (specifically, for the transfer1 packet).

Here are the series of events which lead up to the error.

At stream-start, the SIRM info is acquired. Note that the overall payload in this case is a 1920x1080 bayer image with some small amount of chunk data -- so (1920*1080 + 48 = 2073648):

[14:00:24.953] 🅸 stream> SIRM_INFO             = 0x06000000
[14:00:24.954] 🅸 stream> SIRM_REQ_PAYLOAD_SIZE = 0x00000000001fa430  (2073648)
[14:00:24.954] 🅸 stream> SIRM_REQ_LEADER_SIZE  = 0x00000034
[14:00:24.954] 🅸 stream> SIRM_REQ_TRAILER_SIZE = 0x00000024
[14:00:24.954] 🅸 stream> Required alignment    = 64

The SIRM stream parameters (payload_size * payload_count + final1 + final2) are then set:

[14:00:24.987] 🅸 stream> SIRM_PAYLOAD_SIZE     = 0x00100000 (1048576)     
[14:00:24.987] 🅸 stream> SIRM_PAYLOAD_COUNT    = 0x00000001
[14:00:24.987] 🅸 stream> SIRM_TRANSFER1_SIZE   = 0x000fa440 (1025088) 
[14:00:24.988] 🅸 stream> SIRM_TRANSFER2_SIZE   = 0x00000000
[14:00:24.988] 🅸 stream> SIRM_MAX_LEADER_SIZE  = 0x00000040
[14:00:24.988] 🅸 stream> SIRM_MAX_TRAILER_SIZE = 0x00000040

Note that my device implementation will use these parameters, and write exactly that amount to the host. And so in this case, it will:

1. write 1048576 bytes
2. write 1025088 bytes

Here is an excerpt of the host-side log:

[14:00:27.706] 🅳 sp> Asking for 1048576 bytes
[14:00:27.795] 🅳 sp> Received 52 bytes
[14:00:27.796] 🅳 misc> [PixelFormat::to_gst_caps_string] 0x02180014 -> video/x-raw, format=(string)RGB
[14:00:27.796] 🅳 sp> packet_type  = leader
                     size         = 52
                     frame id     = 1
                     payload_type = extended chunk
                     pixel format = video/x-raw, format=(string)RGB
                     width        = 1920
                     height       = 1080
                     x_offset     = 0
                     y_offset     = 0
[14:00:27.796] 🅳 sp> Asking for 1048576 bytes
[14:00:27.828] 🅳 sp> Received 1048576 bytes
[14:00:27.828] 🅳 sp> packet_type  = image
[14:00:27.828] 🅳 sp> Asking for 1025072 bytes
[14:00:27.855] 🆆 sp> USB transfer error: LIBUSB_ERROR_OVERFLOW
[14:00:27.855] 🅳 sp> Asking for 1025072 bytes
[14:00:27.856] 🅳 sp> Received 36 bytes
[14:00:27.856] 🅳 sp> packet_type  = trailer
                     size         = 36
                     frame id     = 1
                     payload_size = 2073648
[14:00:27.856] 🅳 stream-thread> Received 1048576 bytes - expected 2073648

It seems that aravis on the host-side is happy with the first transfer (of 1048576), but on the second one we receive this LIBUSB_ERROR_OVERFLOW error. I am sort of guessing that this is due to aravis "Asking for 1025072 bytes" here, while the device-side writes 1025088 bytes (as specified in the up-front SIRM parameters, TRANSFER1_SIZE).

I understand where the stream thread derived 1025072 from -- it's simply the remainder of the payload that hasn't been received yet after receiving the payload_count x payload_size packets (SIRM_REQ_PAYLOAD_SIZE - SIRM_PAYLOAD_SIZE*SIRM_PAYLOAD_COUNT). On the other hand, it seems like it should allow the device to send exactly what it was instructed to in those SIRM stream parameters.

Anyway, I wanted to get your quick thoughts on this -- Thanks very much in advance!

[EmmanuelP]

Hi Ryan,

Aravis being developed by reverse engineering, there is surely part of the code that is not correct with respect to the standard.

The different transfer sizes set by aravis in the SIRM registers are rounded using an alignment property from the SIRM_INFO register. Which means the total payload transfer computed from the SIRM registers is greater than the actual image+chunk data size.

I guess Aravis should use what it has set in the SIRM registers when asking for bulk transfers, instead of the actual payload size it expects to fit in the ArvBuffer memory.

Until now, I did not have any report of cameras not working with Aravis, but that not means the code is correct.

If you are willing to, you may try to modify aravis to use the SIRM registers. It should be simple enough. You just have to be careful Aravis is writing directly in the ArvBuffer memory, so the last transfer should happen in the incoming_buffer, then copied to the ArvBuffer memory.

[EmmanuelP]

Aravis being developed by reverse engineering

More correctly, Aravis is developed without using the A3 standard documents. But we use other sources of informations, like the wireshark dissector code, or the NI usb3vision driver code.

[RyanMetcalfeNovanta]

Hi @EmmanuelP, sorry for delay responding. Thanks very much for the information.

If you are willing to, you may try to modify aravis to use the SIRM registers. It should be simple enough. You just have to be careful Aravis is writing directly in the ArvBuffer memory, so the last transfer should happen in the incoming_buffer, then copied to the ArvBuffer memory.

I can try to take a look at this. Thanks for the suggestion!