immersive-web
Current status of binding proposal
Hi there!
I've been looking around (this repo, gpuweb, the WebXR Discord, etc), but I can't seem to find any information on the current status of this proposal.
With the recent stabilisation of WebGPU in Chrome and the Apple Vision Pro supporting WebXR, I'd really like to be able to deploy WebGPU+WebXR content to the web. However, the current version of WebXR only supports WebGL, which limits my options for rendering/existing engine compatibility.
I was going to try a workaround where all rendering is done with WebGPU, and then it's copied into a WebGL context to submit to WebXR, but my understanding is that both cannot be run at the same time on the same page - so that wouldn't work.
Having the ability to use WebGPU with WebXR will unlock a huge amount of potential, especially as more and more standalone headsets are shipping with WebXR support.
I'd appreciate any insight anyone can provide into this ❤️
it seems the draft resolved in a different direction.
Accidental. Definitely will require "cors" for all fetches in the spec.
I understand the security need but it is a shame. That would be one significant boon vs JavaScript solutions.
@domenic Can you explain why CORS is needed? @othermaciej's comment seems reasonable as to why it can be skipped. Of course, if the model is used without CORS, there wouldn't be access to interactivity features.
I tried to explain this in the OP. CORS is a foundational mechanism for enforcing the web's same-origin policy, ensuring that data does not leak across origins without opt-in.
The content of the model file. (Or, the contents of any URL which the attacker puts in <model src="...">.)
The author would never have access to the content of the model file. The UA (or the underlying OS) is responsible for parsing and displaying
The attacker would have access to the contents through side channel attacks, such as Spectre, cache timing attacks, XSLeaks, etc. That is why this is a security concern.
You may be interested in learning more from your security team, or e.g. @annevk, who have been on the forefront of defending from these issues for a long time.
As mentioned, we've already litigated this issue across the web platform many times, most recently on <script type=module> (https://github.com/whatwg/html/issues/1888), with the consensus being to allow no new fetching mechanisms which bypass CORS into the web platform. This is in the process of being codified into TAG guidance, as mentioned in the OP.
The attacker would have access to the contents through side channel attacks, such as Spectre, cache timing attacks, XSLeaks, etc. That is why this is a security concern.
I still don't understand. How is it different from <video>?
We all know that
You may be interested in learning more from your security team, or e.g. @annevk, who have been on the forefront of defending from these issues for a long time.
I don't work for Apple :-)
As mentioned, we've already litigated this issue across the web platform many times, most recently on
<script type=module>(whatwg/html#1888), with the consensus being to allow no new fetching mechanisms which bypass CORS into the web platform. This is in the process of being codified into TAG guidance, as mentioned in the OP.
I agree with @AdaRoseCannon that requiring CORS would make this feature a lot less appealing.
(<video> is problematic and makes ORB more complicated.)
I suppose in part it depends what process the model data ends up in. If it's more like <iframe> and could/will be entirely process-separated I could imagine a solution that does not require CORS, but given the talk in some of the referenced threads about painting to <canvas> that seems unlikely.
I suppose in part it depends what process the model data ends up in. If it's more like
<iframe>and could/will be entirely process-separated I could imagine a solution that does not require CORS, but given the talk in some of the referenced threads about painting to<canvas>that seems unlikely.
<model> is required to be process separated since it renders foreign content and is allowed to interact with browser/OS chrome.
<canvas> was mentioned because it lacks the security/privacy features to implement the proposal.
In addition to the security concerns, if I'm understanding it right, non-CORS third-party content would make it impossible to polyfill the <model> tag since the polyfill script would be unable to access the model resource. That would mean that any site that depends on such content would only work on browsers that natively support <model>, and other browsers wouldn't be able to get any fallback content.
Also, I get the impression that the ability to "hotlink" images and videos (embedding third-party content without consent) is a significant issue for current content, leading to hackish opt-out workarounds such as Referer-based blocking. Since 3D model resource files can be quite large, allowing hotlinking by default could lead to significant bandwidth costs for people who didn't take such countermeasures. I think that instead of expecting people to actively opt out of third-party embedding, it seems cleaner to use CORS as an opt-in approach for a new technology.
<model>is required to be process separated since it renders foreign content and is allowed to interact with browser/OS chrome.
Not sure what you mean by "is allowed to interact with browser/OS chrome"?
In addition to the security concerns, if I'm understanding it right, non-CORS third-party content would make it impossible to polyfill the
<model>tag since the polyfill script would be unable to access the model resource. That would mean that any site that depends on such content would only work on browsers that natively support<model>, and other browsers wouldn't be able to get any fallback content.
Correct. There would be no way to polyfill. Browser that won't implement <mode> would get the poster image.
Also, I get the impression that the ability to "hotlink" images and videos (embedding third-party content without consent) is a significant issue for current content, leading to hackish opt-out workarounds such as Referer-based blocking. Since 3D model resource files can be quite large, allowing hotlinking by default could lead to significant bandwidth costs for people who didn't take such countermeasures. I think that instead of expecting people to actively opt out of third-party embedding, it seems cleaner to use CORS as an opt-in approach for a new technology.
That sounds like a DRM-lite approach. Sites that post models but don't want other sites to use them. Am I reading that correctly?
<model>is required to be process separated since it renders foreign content and is allowed to interact with browser/OS chrome.Not sure what you mean by "is allowed to interact with browser/OS chrome"?
If a model has a reflective surface, its shaders may sample pixels from the UA or the OS. If the device has light estimation, the model can use that to adjust its colors and shadows.
All this is highly privileged info which is why the drawing can't be done by the UA
That sounds like a DRM-lite approach. Sites that post models but don't want other sites to use them. Am I reading that correctly?
No. This is not DRM. This is the same origin policy.
@cabanier wrote:
That sounds like a DRM-lite approach. Sites that post models but don't want other sites to use them. Am I reading that correctly?
It's somewhat orthogonal to DRM. CORS doesn't prevent copying (the model would still be downloadable and could be uploaded somewhere else), but does affect if an asset can be embedded on a page just through a link/reference without needing to be copied first. See the Wikipedia article about inline linking for more background.
I don't have a strong opinion on that aspect (I think the inability to polyfill is more important), and arguably the methods currently used for image/video/audio assets such as Referer-based blocking seem to be working reasonably well in practice, but to me those seem rather clunky and limited by the need to remain backwards compatible.
One additional point is that while the <img> tag does not require CORS, images uploaded as textures to a WebGL canvas do. As images may or may not be embedded into the model resource, this would also affect JS polyfills loading images separately.
I don't have a strong opinion on that aspect (I think the inability to polyfill is more important),
I guess it would still be possible to partially polyfill <mode> in case it's same or cross origin. Other case would only be able to show the poster frame.
A large benefit of having a declarative solution at all is that because of the greater trust it can do things that cannot be done with a JavaScript/WebGL solution. Limiting ourselves to things that can only be accomplished with JS removes some of the point of the whole endeavour.
I think if aspects can't be polyfilled perfectly then it is okay to have a note on the polyfill of the limitations and the developers can work within those constraints until wider support is gained.
Action from the meeting: Request TAG feedback, regarding whether requiring CORS is just the way it is done or if it's still fine to load things style.
Response from TAG: https://github.com/w3ctag/design-reviews/issues/775#issuecomment-1273587963
Response from TAG: w3ctag/design-reviews#775 (comment)
It was strange that they insist on CORS but couldn't give a reason (unless I completely missed it). @annevk said it's a bunch of work but that should be fine considering how much work we'd save for authors.
I don't see much of a point to continue the discussion there because they rendered their decision and clearly don't want to hear from me. I do find it concerning that the TAG states that freely linking to images and videos are "legacy behavior" and "mistakes".
3D models tend to be large files. If anyone can easily embed models hosted on my domain, I'm likely to incur very real bandwidth costs from that. So personally (as a web developer), I feel more comfortable with <model/> requiring CORS, and giving me control as owner of a domain.
3D models tend to be large files. If anyone can easily embed models hosted on my domain, I'm likely to incur very real bandwidth costs from that. So personally (as a web developer), I feel more comfortable with
<model/>requiring CORS, and giving me control as owner of a domain.
I can understand that but:
- the internet is based/designed on freely linking to public information and it is up to us browser developers to make sure that this is safe. If you don't want to share, you have to hide your assets behind a login.
- that was never brought up as a reason. Instead I get a "because security" response.
Just offering my preference as a web developer, I have no comment on what TAG's motivation might be.
Linking to public information is one thing. I want the content to be public on my domain. I don't want to pay for the bandwidth required if the content is embedded on domains X, Y, and Z, and I don't want to put my site behind a login wall to prevent that. Others can download the media and rehost it if they choose. I can enable CORS if I prefer, I do value having that choice.
We had a somewhat inconclusive discussion about the embedding / bandwidth concerns earlier in this issue, see for example https://github.com/immersive-web/model-element/issues/56#issuecomment-1226452989 . The way I understand it, a login wall should not be necessary. Currently, if you want control over embedding on third-party sites for <img> or <video> files, you can configure your web server to check the Referer: header and match it against an allowlist of domains, and ensure that all authorized embedding sites send proper Referer: headers.
I still think that this seems a bit clunky, but I get the impression that in practice people seem to have learned to live with it for the pre-existing media file types, though I'd suspect that in some cases this only happened after unwelcome surprise bandwidth charges.