MediaStream API should support pan, zoom and tilt

[T1mmos]

⚙ Compilation target

ES2022

⚙ Library

TypeScript 5.3.2

Missing / Incorrect Definition

The interfaces MediaTrackCapabilities and MediaTrackConstraints (MediaTrackConstraintSet) should include the zoom, tilt and pan properties, so control of the camera is accessible via TypeScript.

In pure JavaScript, this works: https://googlechrome.github.io/samples/image-capture/update-camera-zoom.html

In TypeScript, the necessary properties for camera control are unavailable: https://github.com/microsoft/TypeScript/blob/main/src/lib/dom.generated.d.ts (e.g. search for "zoom" -> no results).

Context: a use case where (optical) zoom is required is where a user needs to scan a very tiny QR code (1 cm2). If you move your phone too far away, it's too small to scan; if you move too close, the camera cannot focus on it. A solution here is to move your phone further away but allow the user to zoom which would 1. enable the camera to focus and 2. make the QR code big enough in screen space. I noticed that on my phone, the native photo app zooms by default, no troubles scanning the QR code here.

Sample Code

interface MediaTrackCapabilities {
    aspectRatio?: DoubleRange;
    autoGainControl?: boolean[];
    channelCount?: ULongRange;
    deviceId?: string;
    displaySurface?: string;
    echoCancellation?: boolean[];
    facingMode?: string[];
    frameRate?: DoubleRange;
    groupId?: string;
    height?: ULongRange;
    noiseSuppression?: boolean[];
    sampleRate?: ULongRange;
    sampleSize?: ULongRange;
    width?: ULongRange;
}

interface MediaTrackConstraintSet {
    aspectRatio?: ConstrainDouble;
    autoGainControl?: ConstrainBoolean;
    channelCount?: ConstrainULong;
    deviceId?: ConstrainDOMString;
    displaySurface?: ConstrainDOMString;
    echoCancellation?: ConstrainBoolean;
    facingMode?: ConstrainDOMString;
    frameRate?: ConstrainDouble;
    groupId?: ConstrainDOMString;
    height?: ConstrainULong;
    noiseSuppression?: ConstrainBoolean;
    sampleRate?: ConstrainULong;
    sampleSize?: ConstrainULong;
    width?: ConstrainULong;
}

interface MediaTrackConstraints extends MediaTrackConstraintSet {
    advanced?: MediaTrackConstraintSet[];
}

Documentation Link

No response

[IllusionMH]

Is it supported anywhere except Chrome? See https://github.com/microsoft/TypeScript-DOM-lib-generator?tab=readme-ov-file#why-is-my-fancy-api-still-not-available-here which notes that at least 2 different engines should support it to be included.

You can expand those types for your project using declaration emerging.

[RyanCavanaugh]

This API appears to be quite old so I'd assume it's well-supported by now https://github.com/webrtc/samples/blob/gh-pages/src/content/getusermedia/pan-tilt-zoom/js/main.js#L29

[fatcerberus]

declaration emerging

Is this the new TS feature where types for stuff just miraculously appear from the aether?