mediacapture-transform
mediacapture-transform copied to clipboard
w3c
MediaStreamTrack Insertable Media Processing using Streams
I have not been able to find if there is any specification about what is the value of the `timestamp` attribute of an `VideoFrame` or `AudioFrame` from a remote MediaStreamTrack...
`VideoTrackGenerator` (the successor to MSTG) converts a stream of `VideoFrame`s to a `MediaStreamTrack`. `VideoFrame`s may have been "transformed" by a process that generates jitter. For example, they may have been...
At the April 2022 WEBRTC WG Virtual Interim, conversion of `VideoFrame` to WebGPU and back was identified as a performance bottleneck: https://docs.google.com/presentation/d/15iAIhzpaA6reKJBL-ecgYtic6ZKHEpKL5OK_sExTllc/edit#slide=id.g12073675a7a_0_0 Reference: https://github.com/w3c/webcodecs/pull/412
Is it valid to append mutliple VideoFrames or AudioData objects with the same timestamp (e.g. timestamp = 0) to a MediaStreamTrack? If so, what is the behavior? Does the spec...
My recollection from past discussions is that feedback was provided that maxBufferSize is probably unneeded and potentially harmful for processing video frames (there is some discussion https://github.com/w3c/mediacapture-transform/issues/69#issuecomment-838436099 at least). As...
I thought we agreed on adding a reference in the spec to https://github.com/whatwg/streams/blob/main/streams-for-raw-video-explainer.md to explain that the current use of streams requires upgrading streams spec along the lines of the...
Mediacapture-Transform's MSTProcessor interface uses `VideoFrame` . My understanding is that this represents a GPU buffer, but the relationship to [WebGPU's GPUBuffer](https://gpuweb.github.io/gpuweb/#gpu-device) is unclear to me. [WebGPU](https://gpuweb.github.io/gpuweb/), is going into [Origin...
MediaStreamTrackGenerator does not specifiy any limits as to the speed at which data can be fed in. This can lead to problems if developers push 60s worth of audio data...
When obtaining live camera data in native apps, the data is provided in the sensor-native orientation. There's usually an API to determine the orientation of the sensor relative to the...
The current proposed API is based on ReadableStream of frames. It does not seem that pros and cons of this approach have been documented. It would also be interesting to...