mediacapture-extensions
mediacapture-extensions copied to clipboard
Face Detection: How metadata should be tied to MediaStreamTrack video frames
Following on https://github.com/w3c/mediacapture-extensions/pull/69 and media capture transform, face detection metadata could be made available to mediastreamtrack transforms. There are a few possibilities we could envision. The following come to mind:
- Attach FaceDetection metadata to VideoFrame with dedicated face detection metadata getter/setter (new VideoFrame slot that can be cloned/postMessaged).
- Attach FaceDetection metadata to VideoFrame using a generic metadata mechanism (mechanism to be defined, see VideoTrackGenerator).
- Make MediaStreamTrack transforms expose objects that have a VideoFrame and a metadata object.
- Extend MediaStreamTrackProcessor.readable with a face detection metadata getter (related to the last read video frame). And VideoTrackGenerator.writable with a metadata setter.