Accelerated Shape Detection in Images

[reillyeon]

WebKittens

No response

Title of the spec

Accelerated Shape Detection in Images

URL to the spec

https://wicg.github.io/shape-detection-api/

URL to the spec's repository

https://github.com/WICG/shape-detection-api

Issue Tracker URL

No response

Explainer URL

No response

TAG Design Review URL

https://github.com/w3ctag/design-reviews/issues/176

Mozilla standards-positions issue URL

https://github.com/mozilla/standards-positions/issues/21

WebKit Bugzilla URL

https://bugs.webkit.org/show_bug.cgi?id=198174

Radar URL

rdar://51089129

Description

The Accelerated Shape Detection in Images specification is an old incubation in the WICG which has been partially shipped in Chromium-based browsers. Specifically, the BarcodeDetector API is available, while the FaceDetector and TextDetector APIs are still behind a flag. The reason for this is that we (Google) felt that support for barcode detection was sufficiently available across our supported platforms that the API was ready to ship, while the face and text detection capabilities were not as consistent and so more development was required.

I've noticed that changes related to this specification have started to land in WebKit and so it seems prudent to ask for a formal standards position and if Apple is interested in graduating this specification out of the WICG and into a Working Group.

[hober]

Added concerns: venue because this should (probably, IMO) be taken up by the WebML group.

[reillyeon]

Agreed, if Apple is interested in collaborating on this spec that's a signal to me that it should graduate from incubation and the WebML WG seems like a good venue.

[marcoscaceres]

This is a DRAFT POSITION and does not represent the consensus of the WebKit community. However, unless anyone objects by 17th of May we will make the below the "official position". Please feel free to object or request more time if needed. Please expressed support through the thumbs up reaction or additional comments (no +1 comments please!).

Discussing with our colleagues internally, we are “supportive” of this standardization effort with some general concerns:

• Initially restricting the scope of this standardization effort to barcodes and faces is reasonable - standardizing around faces and barcodes would already provide significant benefits to the Web Platform.
• The spec could use some general fixes (e.g., should be written to use steps, use of task queues to resolve promises could be improved, etc.), but generally the spec gives a good overview about how everything can be implemented.
• While implementing in WebKit we ran into issues with the test suite. We are appreciative that there was an initial test suite, but we ask that folks contributing tests be more mindful to not make tests browser specific.
• Although the test suite will not serve as ML training data, the standardization effort should nonetheless ensure that the test data is representative of people of different ages, genders, races, and ethnicities.
• It's good that this has been started in the WICG first instead of going straight to a Working Group. As we get serious about standardizing this, we'd like to discuss a suitable home for the spec (probably the WebML WG). At the same time, we should consider how this API will interface or overlap with work already underway in the Web ML WG.
• The APIs could perhaps benefit from allowing multiple detection requests into a single method call.
• A significant use case for the face detection aspect is likely to be video streams. As such, the standardization effort should continue to explore how to make the API work effectively with such streams. At the same time, we are interested in hearing from the web developer community about additional potential uses for this API.

We look forward to collaborating further on this API with the WICG community and working towards standardization.

[marcoscaceres]

Removing venue as a concern, as we have to possibility to move it somewhere more suitable.

[litherum]

(By the way, I started a draft implementation of this for the Apple ports of WebKit. It's currently behind an experimental feature, off by default.)

[Tachi107]

Hi all, according to https://github.com/WebKit/standards-positions/issues/174#issuecomment-1539415805 Webkit's position should be supporting of a subset of this new API, but looking at https://webkit.org/standards-positions/ I can't find any reference to the Shape Detection API. Has Webkit's stance changed since the mentioned comment? Thanks for your efforts!

[annevk]

Seems like you missed this part:

This is a DRAFT POSITION and does not represent the consensus of the WebKit community.

[Tachi107]

Il giorno dom 25 feb 2024 alle 04:09:39 -08:00:00, Anne van Kesteren @.***> ha scritto:

Seems like you missed this part:

This is a DRAFT POSITION and does not represent the consensus of the WebKit community.

Hi Anne, thanks for your reply! I saw that, but interpreted the “However, unless anyone objects by 17th of May we will make the below the "official position"” part as having the intention of promoting the cited position from draft status to official by the mentioned date. Sorry for my misinterpretation!

[marcoscaceres]

Yeah, we forgot to close off the position as no one objected. Labeled as "support".