Provide a way to map POI to particular camera option (WB, AE, Focus)

[alexshalamov]

Image Capture API specifies list point of interests, where each point (it seems) is responsible for running 3A algorithms. All other OS platform APIs allow to specify POI / area separately. Providing way to map POI to AE / Focusing / WB, would align API with platform interfaces and provide better web developer user experience.

Also, at the moment, it is unclear how setting multiple points would affect resulted capture. Should UA take average between points, or only calculate settings from focused POI?

[yell0wd0g]

All other OS platform APIs allow to specify POI / area separately.

UVC 1.5 only allows for 1 Region of Interest (see Section 4.2.2.1.20), and the user can only switch the auto-settings on/off in it. This pretty much covers the whole Desktop world and iOS.

In Android-land, in my experience the use of separate POIs for any of the 3A algorithms is marginal and just doesn't justify the hassle of extending this API, in other words, how often have you seen anyone configuring different POIs to be used for auto white balance and, say, AE & AF? The vast majority of camera apps wire the three together upon user tapping on the preview viewport (the user still has another layer of control via disabling any of the three "auto" settings).

Moreover the support for several POIs in Android (e.g. here) leaves plenty of latitude to the hardware as to how to implement several POIs / overlapping areas of interest, because the API contract is weak (and the OEMs protect their algorithms).

If deemed interesting I could make a note/link in the Spec explaining all this.

[alexshalamov]

iOS, Android and Windows support focus point per parameter, that will cover >85% mobile use cases. Browser usage on Windows desktops is > 90% market share. If underlying platforms support that feature, why not to expose it for web developers? Spec could provide ‘default’ behavior for POI settings, e.g., all points would be responsible for 3A, yet, leave possibility for developers to provide fine-grained settings. What do you think?

[yell0wd0g]

Apologies, there was a mistaken link in my previous comment (corrected), UVC 1.5. should point to https://tinyurl.com/uvc-1-5-specification which is an uploaded copy of the Spec :-)

The relevant part:

4.2.2.1.20 Digital Region of Interest (ROI) Control

The rectangle specified

Note that there's only one such rectangle. Also the table 4-28 has a bitmap bmAutoControls that allows the user to configure which algorithm(s) to connect to the only available ROI. (This was somehow lost in my previous comment).

[alexshalamov]

@yellowdoge yes, UVC support 1 POI that can be configured to control WB, AE, focus or other parameters (separately or simultaneously).

Currently Image Capture API already supports multiple POIs, we can improve it and add fine grained feature control per POI.

This will be beneficial for most of the mobile devices (iOS, Android, Windows) as well as some desktop platforms (Windows, CrOS?). To keep backward compatibility, if parameter is not explicitly set for POI, we can assign POI to control all parameters.

For platforms that do not support multiple POIs, we can modify constrants, for example:

partial dictionary MediaTrackSupportedConstraints {
...
  // From boolean => unsigned short
  // boolean pointsOfInterest = true;
  unsigned short pointsOfInterest = 1;
...
}

What do you think?