esp-iot-solution
esp-iot-solution copied to clipboard
espressif
[usb_stream] Bad UVC frame (AEGHB-659)
Answers checklist.
- [X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
- [X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
- [X] I have searched the issue tracker for a similar issue and not found a similar issue.
General issue report
I have successfully connected ESP32-S3 (N16R8) to a UVC camera using the example usb_camera_mic_spk, but the obtained frame looks very bad as below:
Related logs
I (643) USB_STREAM: line 151 HCD_PORT_EVENT_CONNECTION
I (644) USB_STREAM: Action: ACTION_DEVICE_CONNECT
I (694) USB_STREAM: Resetting Port
I (754) USB_STREAM: Setting Port FIFO, 1
I (755) USB_STREAM: USB Speed: full-speed
I (755) USB_STREAM: ENUM Stage START, Succeed
I (756) USB_STREAM: ENUM Stage GET_SHORT_DEV_DESC, Succeed
I (768) USB_STREAM: Default pipe endpoint MPS update to 64
I (768) USB_STREAM: ENUM Stage CHECK_SHORT_DEV_DESC, Succeed
I (769) USB_STREAM: ENUM Stage SET_ADDR, Succeed
I (791) USB_STREAM: ENUM Stage CHECK_ADDR, Succeed
I (792) USB_STREAM: ENUM Stage GET_FULL_DEV_DESC, Succeed
*** Device descriptor ***
bcdUSB 2.00
bDeviceClass 0xef
bDeviceSubClass 0x2
bDeviceProtocol 0x1
bMaxPacketSize0 64
idVendor 0x2bdf
idProduct 0x101
bNumConfigurations 1
I (805) USB_STREAM: ENUM Stage CHECK_FULL_DEV_DESC, Succeed
I (806) USB_STREAM: ENUM Stage GET_SHORT_CONFIG_DESC, Succeed
I (818) USB_STREAM: ENUM Stage CHECK_SHORT_CONFIG_DESC, Succeed
I (819) USB_STREAM: ENUM Stage GET_FULL_CONFIG_DESC, Succeed
*** Configuration descriptor ***
wTotalLength 433
bNumInterfaces 2
bConfigurationValue 1
*** Interface Association Descriptor: Video ***
*** Interface descriptor ***
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 0xe (Video)
bInterfaceSubClass 0x1
*** Endpoint descriptor ***
bEndpointAddress 0x83 EP 3 IN
bmAttributes 0x3 INT
wMaxPacketSize 16
bInterval 8
*** Interface descriptor ***
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 0xe (Video)
bInterfaceSubClass 0x2
*** Class-specific VS Interface Descriptor ***
bNumFormats 3
*** VS Format MJPEG Descriptor ***
bFormatIndex 0x2
bNumFrameDescriptors 3
bDefaultFrameIndex 1
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x1
wWidth 120
wHeigh 160
FrameInterval[0] 400000
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x2
wWidth 240
wHeigh 320
FrameInterval[0] 333333
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x3
wWidth 640
wHeigh 360
FrameInterval[0] 333333
*** Endpoint descriptor ***
bEndpointAddress 0x81 EP 1 IN
bmAttributes 0x2 BULK
wMaxPacketSize 64
bInterval 0
I (826) USB_STREAM: Actual VS Interface(MPS <= 512) found, interface = 1, alt = 0
I (827) USB_STREAM: Endpoint(BULK) Addr = 0x81, MPS = 64
I (827) USB_STREAM: Actual MJPEG format index, format index = 2, contains 3 frames
I (828) USB_STREAM: Actual Frame: 7, width*heigh: 65535*65535, frame index = 1
I (828) USB_STREAM: ENUM Stage CHECK_FULL_CONFIG_DESC, Succeed
I (830) USB_STREAM: ENUM Stage SET_CONFIG, Succeed
I (832) uvc_mic_spk_demo: UVC: get frame list size = 3, current = 0
I (832) uvc_mic_spk_demo: frame[0] = 120x160
I (833) uvc_mic_spk_demo: frame[1] = 240x320
I (833) uvc_mic_spk_demo: frame[2] = 640x360
I (833) uvc_mic_spk_demo: Device connected
I (834) USB_STREAM: Probe Format(2), Frame(1) 120*160, interval(400000)
I (834) USB_STREAM: Probe payload size = 1024
bFormatIndex: 2
bFrameIndex: 1
dwFrameInterval: 400000
dwMaxPayloadTransferSize: 16384
bInterfaceNumber: 0
I (865) USB_STREAM: dwMaxPayloadTransferSize set = 1024, probed = 16384
I (865) USB_STREAM: Sample processing task start
I (886) USB_STREAM: USB Device Connected
W (10546) USB_STREAM: line 155 HCD_PORT_EVENT_DISCONNECTION
I (10547) USB_STREAM: Recover Stream Task
I (10547) USB_STREAM: usb stream task wait reset
I (10547) USB_STREAM: usb stream task reset, reason: device recover
I (10547) USB_STREAM: Resetting UVC pipe
I (10547) USB_STREAM: sample processing stop
I (10548) USB_STREAM: Sample processing task deleted
I (10598) uvc_mic_spk_demo: Device disconnected
I (10599) USB_STREAM: Action: ACTION_PIPE_DFLT_DISABLE
I (10599) USB_STREAM: Action: ACTION_PORT_RECOVER
I (10599) USB_STREAM: Action: ACTION_DEVICE_DISCONN
I (10599) USB_STREAM: Waiting USB Connection
I (45973) USB_STREAM: line 151 HCD_PORT_EVENT_CONNECTION
I (45974) USB_STREAM: Action: ACTION_DEVICE_CONNECT
I (46024) USB_STREAM: Resetting Port
I (46084) USB_STREAM: Setting Port FIFO, 1
I (46084) USB_STREAM: USB Speed: full-speed
I (46085) USB_STREAM: ENUM Stage START, Succeed
I (46086) USB_STREAM: ENUM Stage GET_SHORT_DEV_DESC, Succeed
I (46088) USB_STREAM: Default pipe endpoint MPS update to 64
I (46088) USB_STREAM: ENUM Stage CHECK_SHORT_DEV_DESC, Succeed
I (46089) USB_STREAM: ENUM Stage SET_ADDR, Succeed
I (46100) USB_STREAM: ENUM Stage CHECK_ADDR, Succeed
I (46101) USB_STREAM: ENUM Stage GET_FULL_DEV_DESC, Succeed
*** Device descriptor ***
bcdUSB 2.00
bDeviceClass 0xef
bDeviceSubClass 0x2
bDeviceProtocol 0x1
bMaxPacketSize0 64
idVendor 0x2bdf
idProduct 0x101
bNumConfigurations 1
I (46103) USB_STREAM: ENUM Stage CHECK_FULL_DEV_DESC, Succeed
I (46104) USB_STREAM: ENUM Stage GET_SHORT_CONFIG_DESC, Succeed
I (46106) USB_STREAM: ENUM Stage CHECK_SHORT_CONFIG_DESC, Succeed
I (46107) USB_STREAM: ENUM Stage GET_FULL_CONFIG_DESC, Succeed
*** Configuration descriptor ***
wTotalLength 433
bNumInterfaces 2
bConfigurationValue 1
*** Interface Association Descriptor: Video ***
*** Interface descriptor ***
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 0xe (Video)
bInterfaceSubClass 0x1
*** Endpoint descriptor ***
bEndpointAddress 0x83 EP 3 IN
bmAttributes 0x3 INT
wMaxPacketSize 16
bInterval 8
*** Interface descriptor ***
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 0xe (Video)
bInterfaceSubClass 0x2
*** Class-specific VS Interface Descriptor ***
bNumFormats 3
*** VS Format MJPEG Descriptor ***
bFormatIndex 0x2
bNumFrameDescriptors 3
bDefaultFrameIndex 1
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x1
wWidth 120
wHeigh 160
FrameInterval[0] 400000
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x2
wWidth 240
wHeigh 320
FrameInterval[0] 333333
*** VS MJPEG Frame Descriptor ***
bFrameIndex 0x3
wWidth 640
wHeigh 360
FrameInterval[0] 333333
*** Endpoint descriptor ***
bEndpointAddress 0x81 EP 1 IN
bmAttributes 0x2 BULK
wMaxPacketSize 64
bInterval 0
I (46136) USB_STREAM: Actual VS Interface(MPS <= 512) found, interface = 1, alt = 0
I (46136) USB_STREAM: Endpoint(BULK) Addr = 0x81, MPS = 64
I (46136) USB_STREAM: Actual MJPEG format index, format index = 2, contains 3 frames
I (46137) USB_STREAM: Actual Frame: 7, width*heigh: 65535*65535, frame index = 1
I (46137) USB_STREAM: ENUM Stage CHECK_FULL_CONFIG_DESC, Succeed
I (46139) USB_STREAM: ENUM Stage SET_CONFIG, Succeed
I (46141) uvc_mic_spk_demo: UVC: get frame list size = 3, current = 0
I (46141) uvc_mic_spk_demo: frame[0] = 120x160
I (46142) uvc_mic_spk_demo: frame[1] = 240x320
I (46142) uvc_mic_spk_demo: frame[2] = 640x360
I (46142) uvc_mic_spk_demo: Device connected
I (46143) USB_STREAM: Probe Format(2), Frame(1) 120*160, interval(400000)
I (46143) USB_STREAM: Probe payload size = 1024
bFormatIndex: 2
bFrameIndex: 1
dwFrameInterval: 400000
dwMaxPayloadTransferSize: 16384
bInterfaceNumber: 0
I (46174) USB_STREAM: dwMaxPayloadTransferSize set = 1024, probed = 16384
I (46174) USB_STREAM: Sample processing task start
I (46218) uvc_mic_spk_demo: uvc callback! frame_format = 7, seq = 1, width = 120, height = 160, length = 5391, ptr = 0
// ommited some lines of invalid frame (length too short)
// valid frame starts below:
I (47767) uvc_mic_spk_demo: uvc callback! frame_format = 7, seq = 14, width = 120, height = 160, length = 16734, ptr = 0
I (47783) uvc_mic_spk_demo: uvc callback! frame_format = 7, seq = 15, width = 120, height = 160, length = 16708, ptr = 0
I (47811) uvc_mic_spk_demo: uvc callback! frame_format = 7, seq = 16, width = 120, height = 160, length = 16746, ptr = 0
// ...
Since the camera is the same as the one in https://github.com/espressif/esp-iot-solution/issues/325, I have set CONFIG_NUM_BULK_BYTES_PER_URB=1024 to make it work.
It seems to be caused by:
https://github.com/espressif/esp-iot-solution/blob/f2a2669baf50d499b0890b71cd7b03fc9a536315/components/usb/usb_stream/usb_stream.c#L2041-L2043
which is setting data_len incorrectly. I'm getting mostly correct frames after changing this part to:
} else if (strmh->reassembling) {
header_len = payload[0];
data_len = payload_len - header_len;
/* checking the end-of-header */
variable_offset = 2;
header_info = payload[1];
ESP_LOGV(TAG, "reassembling %u + %u", strmh->got_bytes, data_len);
} else {
But I'm still getting invalid frames constantly (around every 150 frames), I guess this function still needs more fixes.
Most USB Bulk cameras use Single Transfer per Sample mode, which is usb_stream currently supported.
The camera you currently use seems to be using Multiple Transfers per Sample mode, which is not supported now.
The camera you currently use seems to be using
Multiple Transfers per Samplemode, which is not supported now.
Thanks for the clue! Are you planning to support this?
@zry98 It has been supported in https://github.com/espressif/esp-iot-solution/pull/361, please help try it with your camera.
Unfortunately, it still requires the changes in https://github.com/espressif/esp-iot-solution/issues/360#issuecomment-2119383785 to work. Can you test if https://github.com/zry98/esp-iot-solution/commit/d3fa9629714ac5b7e0aac0811334c4dd9c6779d0 is OK for cameras of single transfer per sample mode? (I don't have one to test against)
Btw, I noticed that with the changes applied, it randomly panics after some time as below:
Guru Meditation Error: Core 0 panic'ed (Interrupt wdt timeout on CPU0).
Core 0 register dump:
PC : 0x4037ab9a PS : 0x00060934 A0 : 0x82004365 A1 : 0x3fc9f030
0x4037ab9a: esp_cpu_wait_for_intr at C:/tools/esp-idf/v5.2.1/esp-idf/components/esp_hw_support/cpu.c:145
A2 : 0x00000000 A3 : 0x00000001 A4 : 0x8037e7b6 A5 : 0x3fce8340
A6 : 0x00060023 A7 : 0x00000003 A8 : 0x8201cc0e A9 : 0x3fc9f000
A10 : 0x00000000 A11 : 0x00000001 A12 : 0x8037e7b6 A13 : 0x3fc9f1e0
A14 : 0x3fc9abec A15 : 0x00000001 SAR : 0x0000001e EXCCAUSE: 0x00000005
EXCVADDR: 0x00000000 LBEG : 0x00000000 LEND : 0x00000000 LCOUNT : 0x00000000
Backtrace: 0x4037ab97:0x3fc9f030 0x42004362:0x3fc9f050 0x4037f50c:0x3fc9f070
0x4037ab97: xt_utils_wait_for_intr at C:/tools/esp-idf/v5.2.1/esp-idf/components/xtensa/include/xt_utils.h:81
(inlined by) esp_cpu_wait_for_intr at C:/tools/esp-idf/v5.2.1/esp-idf/components/esp_hw_support/cpu.c:132
0x42004362: esp_vApplicationIdleHook at C:/tools/esp-idf/v5.2.1/esp-idf/components/esp_system/freertos_hooks.c:59
0x4037f50c: prvIdleTask at C:/tools/esp-idf/v5.2.1/esp-idf/components/freertos/FreeRTOS-Kernel/tasks.c:4273 (discriminator 1)
@zry98 https://github.com/zry98/esp-iot-solution/commit/d3fa9629714ac5b7e0aac0811334c4dd9c6779d0 does not work in our test cameras
According to the UVC1.5 protocol, One payload only contains one header. So we can't subtract the length of the header from each packet. If your camera only works this, it may be a custom protocol, can you provide a link to buy the camera?
for the random panics, it could has no direct connection with the patch. Would you please run the example under this repository to test your camera, and please config your log level to verbose to get more details
https://github.com/zry98/esp-iot-solution/commit/d3fa9629714ac5b7e0aac0811334c4dd9c6779d0 does not work in our test cameras
@leeebo Sorry I didn't pay much attention to the header, thought a simple if (bulk_xfer) would do it.
So we still need to find a way to know if it's in single or multi transfer mode, maybe checking the first several transfers to determine? then process the later ones accordingly.
can you provide a link to buy the camera?
It's the Hikvision DS-2TM01-3XF (or TB-4117-3/S for OEM, more popular on the market) thermal camera. （在淘宝和闲鱼能找到很多）
PotPlayer, webcamtests.com on Windows and USB Camera on Android all work flawlessly with it.
For your reference, I've captured some packets with Wireshark: hik4117.zip
Hi @zry98 would you please send us the camera for better support? After the fix, we will send it back to you.