openscreenprotocol
openscreenprotocol copied to clipboard
w3c
Use the same codec names as the media APIs.
MediaCapabilities (and other media APIs) uses a content type of "video/foo; codecs=A.B.C". We should probably use the "A.B.C" instead of what we have currently. For example, instead of "h264", use "avc1.4200" or instead of "vp9", "vp09.00.10.08". It's a little crazy and I can't find any RFCs for these things, but here is where Chromium parses them:
https://cs.chromium.org/chromium/src/media/base/mime_util_internal.cc?g=0&l=820
Is this what you are looking for? It covers some, but not all of the MPEG codecs.
https://tools.ietf.org/html/rfc6381
There may be an equivalent RFC for the VP*/AV1 codecs.
The media APIs don't make any explicit reference to any RFCs. I have no idea what implementers are supposed to do, other than copy each other's source code :-P
Spoke with Dan Sanders, there are other RFCs for the webm container we can refer to as well.
He noted that enumeration of codec strings for any given codec family is a hard problem because of the incredibly rich syntax for codecs - so we should avoid that in our protocol design.
Take a look at these specs for VPX and AV1 and you'll see what I mean:
https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp-codec-iso-media-file-format-binding-20160516-draft.pdf
https://aomediacodec.github.io/av1-isobmff/#codecsparam
It sounds like there wasn't an explicit action recorded in the last F2F minutes [1], but for the record, we discussed consulting with the Media WG folks working on Media Capabilities and related APIs to get feedback on this issue.
[1] https://www.w3.org/2021/02/23-webscreens-minutes.html#t02
The minutes from the joint meeting with Media WG are here. Summary:
- We can use references in the WebCodecs registry for codec string details: https://www.w3.org/TR/webcodecs-codec-registry/
Want to discuss briefly and propose putting up a pull request to update the specification to explicitly use the WebCodecs registry to resolve this issue.
@chrisn, at the F2F [*] you mentioned you would follow up with any thoughts on how to extend the codecs name list beyond what is defined through WebCodecs. Maybe we can just fall back on the language that is already there, which is:
This must be a single-codec RFC 6381 codecs parameter
for codecs not listed in WebCodecs.
[*] https://www.w3.org/2022/06/08-webscreens-minutes.html#t08
Yes, that could work, but it doesn't really help to know how to implement. Is a fall back enough? If we do need to be more specific, then we'd need to find relevant specs which aren't in the WebCodecs registry, e.g., via MP4RA - and perhaps turn that into its own spec, which as you noted earlier is a hard problem.
Good point. We could provide guidance along the lines that the codec string should only be returned in the capabilities response if MediaCapabilities would return "supported" and "smooth" in combination with other relevant parameters.
need to find relevant specs which aren't in the WebCodecs registry, e.g., via MP4RA
I'm not familiar with that registry; it looks like it's describing how media is described by MP4 containers? Handling the details container formats is (right now) out of scope for OSP.
It seems like the alternatives for describing these strings are the MIME standards, or various codec-specific RFCs.
I can put up a PR describing what I have in mind.