Video UVC 1.5 check list

[hathach]

Is your feature request related to a problem? Please describe

UVC is added by #1118 This issue is used to keep track of current status of video class driver on each DCD ports. Once it is all passed, we could close this.

MCU	Video Capture	Note
nRF52840	:heavy_check_mark:
rp2040	:heavy_check_mark:
KL25z	:heavy_check_mark:
SAMD21	:heavy_check_mark:
stm32f411	:heavy_check_mark:

How to test

run examples/device/video_capture

Linux/macOS

not available just yet

WIndows

from https://github.com/hathach/tinyusb/pull/1118#issuecomment-936185184

• Download and run virtualdub
• Launch virtualDub, File -> Capture AVI ...
• File -> Set capture file... Input a file path to save captured video stream.
• Device -> USB (Direct Show)
• Video -> No display
• Capture -> Capture Video
• Wait for a few seconds then click Capture -> Stop Capture
• Check out the capture avi file, it should show the color bar

[me-no-dev]

Hi @hathach :) I tried UVC last night and found errors in the example descriptor. For one the UUID of the format is wrong (can be seen in dmesg on Linux). Then there are more complaints from the kernel. It is not at all working on Mac and on Windows it shows the camera but fails to init. I will do some more looking into if someone does not do it before me :)

[hathach]

@me-no-dev yeah, I saw my Linux machine complaining about the format as well. haven't tried to find the cause just yet. Please let me know if you could figure out why.

[45883.869206] usb 3-4: Product: TinyUSB Device
[45883.869208] usb 3-4: Manufacturer: TinyUSB
[45883.869209] usb 3-4: SerialNumber: 123456
[45883.877143] uvcvideo: Unknown video format 3231564e-0000-0010-0080-719b3800aa00
[45883.877147] uvcvideo: Found UVC 1.50 device TinyUSB Device (cafe:4020)
[45883.877150] uvcvideo: No valid video chain found.

[me-no-dev]

The UUID thing is easy. For example this line should become (swap some bytes). Same for the rest.

//#define TUD_VIDEO_GUID_YUY2   0x59,0x55,0x59,0x32,0x00,0x00,0x10,0x00,0x00,0x80,0x71,0x9B,0x38,0x00,0xAA,0x00
  #define TUD_VIDEO_GUID_YUY2   0x59,0x55,0x59,0x32,0x00,0x00,0x10,0x00,0x80,0x00,0x00,0xAA,0x00,0x38,0x9B,0x71

[kkitayam]

@me-no-dev Thank you for finding the bug! I was misunderstand byte ordering of GUID.

[kkitayam]

uvcvideo: No valid video chain found.

This message is here. I guess the message indicates no input terminal connects with the streaming interface on the chain. This may be related to the Media Controller API.

The configuration descriptor of video_capture has a input terminal as a COMPOSITE_CONNECTOR and the terminal is connected with the streaming interface.

I have looked into Linux kernel drivers and found the followings. COMPOSITE_CONNECTOR is handled by uvc_mc_init_entity(). The uvc_mc_init_entity() is called from uvc_mc_register_entities. And the uvc_mc_register_entities() is called from uvc_register_chains(). The uvc_register_chains() seems to creates a chain. And the function seems to require CONFIG_MEDIA_CONTROLLER in kernel config to call uvc_mc_register_entities(). If CONFIG_MEDIA_CONTROLLER is not enabled in kernel config, CONFIG_MEDIA_CONTROLLER is not registered on a chain and the message (No valid video chain found) will be output.

[xmos-jmccarthy]

Are the video descriptor macros going to be moved to usbd.h in the future?

[hathach]

@xmos-jmccarthy yes, in the future, there will be an UVC template for most common used descriptors. Too bad, I haven't spent much time on UVC recently.

[leeebo]

The commit https://github.com/hathach/tinyusb/pull/1118 works well for ESP32S2/S3 with dcd_dwc2 driver on Windows10 , but seems not work on Linux (libuvc).

[Staacks]

ok, I think this is the right place to give some feedback as I have just published a project using the UVC implementation: An rp2040-based Game Boy adapter that captures the bus communication between the Game Boy and a game cartridge, runs it through an emulator and outputs the gameplay footage video via USB. (More infos: blog article with details, source on github).

So, first of all, thanks a lot for implementing this!

Due to the bandwidth limitation of isochronous transfer on a full speed device I output 160x144 (Game Boy's native resolution) at 29fps in NV12 format - as a compromise between bandwidth and compatibility.

And it works almost everywhere:

• Linux: It just shows up as a webcam and works. Tested in VLC, OBS, ffmpeg, Zoom
• Windows: It just works in OBS, but VLC and Zoom are having problems. VLC complains about the format (which is rather exotic after all) and Zoom freezes, but I would attribute this to difficulties with that format.
• Android: It also works on Android as an external webcam (directly plugging it into the USB-C port of my Pixel 6). Of course one first has to find an app that supports an external camera as source (although Android's API covers that case) and supports the format.
• MacOS: Here I was not able to get any video signal and since I tried ffmpeg, VLC and OBS, which I usually would consider being rather flexible about formats, I think that there might be an actual issue here. But I will have to find some time to play with a minimal example to narrow it down and give further feedback.

So, thanks again for offering UVC!

[Staacks]

Update: I got it to work on a MacBook with Intel CPU by allowing too small wLength values (see this pull request). MacOS sets wLength to 0x22 instead of 0x30 during probe and commit and it does so consistently also with commercial devices.

However, I still have problems on a MacBook with M2 chip. Here the image is green and distorted and by comparing the raw data on the USB bus (which looks correct) with the data that actually reaches ffmpeg on the system, it seems like the system/driver/avfoundation/whatever strips out every other block of 160 bytes.

I have no idea why it should do so and the number 160 only matches the width of the resolution I use and nothing else. But since I use NV12 format and it literally removes every second block of 160 bytes, this also cannot be attributed to a misguided interlacing algorithm. So, with the same device working on the Intel MacBook and the arbitraty removal of data from within individual USB frames, I currently suspect a bug for the Apple silicon devices. Since other webcams work, it is also probably related to the less common NV12 format.

[hathach]

@Staacks that is a brilliant project, thank you for sharing. Regarding the issue with macos, feel free to post/file more issue/discussion or anything you find while troubleshooting. I am sure to help whenever possible.

[beckmx]

Hello guys, I was wondering if with this uvc features, could a pico 2040 connect to a webcam and stream via web?