zephyr icon indicating copy to clipboard operation
zephyr copied to clipboard

usb: device_next: new USB Video Class (UVC) implementation

Open josuah opened this issue 6 months ago • 29 comments

Dependencies:

  • https://github.com/zephyrproject-rtos/zephyr/pull/78603
  • https://github.com/zephyrproject-rtos/zephyr/pull/79168

This is a preview of an USB Video Class implementation for Zephyr that was only tested on custom devices insofar.

Proper Zephyr samples will be provided on upcoming commits. The API is simply to submit frames to the UVC device like to any Zephyr video device.

There is an unsolved challenge around the Video API: there is no set_format because the Zephyr application cannot decide what the host uses, only get_format for what the host does support. But there is a missing video API to allow the driver to warn the application about a forced format change requested by the host. I thought of maybe reusing set_signal() to also warn about format changes and not just buffer events.

	.get_format = uvc_get_format,
	.get_caps = uvc_get_caps,

I will now work on building examples for existing Zephyr devices, as this was built for a custom USB3 board. Here is the devicetree configuration used insofar:

#include <zephyr/dt-bindings/usb/video.h>

&zephyr_udc0 {
	/* IN and OUT counting together as one endpoint */
	num-bidir-endpoints = <2>;

	/* full-speed: 12 Mhz, high-speed: 480 Mhz, super-speed: 5000 Mhz */
	maximum-speed = "high-speed";

	uvc0: zephyr_uvc0 {
		compatible = "zephyr,uvc";
		status = "okay";

		format_uncompressed {
			compatible = "zephyr,uvc-format-uncompressed", "zephyr,uvc-format";
			fourcc = "YUYV";
			guid = [UVC_GUID_YUY2];
			bits-per-pixel = <16>;

			4K {
				size = <3840 2160>;
				max-fps <22>;
			};

			FHD {
				size = <1920 1080>;
				max-fps = <87>;
			};

			VGA {
				size = <640 480>;
				max-fps = <589>;
			};

			qVGA {
				size = <320 240>;
				max-fps = <2347>;
			};
		};
		
		format_mjpeg {
			compatible = "zephyr,uvc-format-mjpeg", "zephyr,uvc-format";

			FHD {
				size = <1920 1080>;
				max-fps = <60>;
			};

			VGA {
				size = <640 480>;
				max-fps = <120>;
			};
		};
	};
};

The sizes and FPS are to be selected by the developer. The USB descriptors get configured as described above, and the host will request a particular format. Once that is done, the USB Video Class driver can let the application know which of these was selected through the video.h get_format() API.

This is still a draft PR, but I am grateful for comments and suggestions. I am willing to do the work of refactoring this as much as needed. For instance:

  • Add usbd_ prefix
  • Use a more sensible formatting than what clang-format does i.e. for structs

josuah avatar Aug 07 '24 14:08 josuah