KHR_anchors_plane - common home AR use cases, equivalent to what USDZ already supports.

[bhouston]

This draft extension aims to make glTF equivalent with USDZ in terms of its ability to internally specify plane anchor points within the home, such as floor/ground, ceiling and wall. It only supports these vertical or horizontal planes, it does not support slopes.

This is a missing capability within glTF.

For example, Google Android gets around it using a series of URL-based "intents" that denote whether the glTF file supports AR floor placement ("ar_preferred") and/or vertical placement ("enable_vertical_placement".). This is Google trying to work around this limitation in glTF. These types of intents are not required when using USDZ in Safari and it is a much smoother process.

[donmccurdy]

Are you aware of the preferredSurfaces property available in the model3d XMP namespace? This is available in glTF now. I don't know whether AR viewers have implemented support or whether modelviewer can currently pass it through when autogenerating a USDZ file for iOS devices.

In my opinion, the advantages of using XMP for AR-related metadata are:

1. AR metadata can be generalized beyond glTF
2. More practical versioning system than extensions / easier updates for rapidly-evolving AR use cases
3. Possibly easier pass-through of XMP metadata in processing tools that aren't AR-related and might not implement an extension

The main limitation of the XMP approach is that while XMP packets can be attached to glTF objects like scenes or nodes, a packet cannot reference other nodes conveniently, e.g. by index.

[bhouston]

I think that the XMP information is good for the basic AR use case for planes/surfaces. Let's go in that direction for this extension. We just have to better publicize it. I think a blog post would be useful on this topic to really push it forward.

The "face" option is incorrect and should be removed from this list. The face is not a surface. It requires additional information to specify putting something on a face correctly. ( I've done a bunch of face try-on work but it isn't public, but if you scroll down there are face virtual try-on examples in gif form: https://www.threekit.com/immersive-shopping-suite. I've tried to codify the simplest method of face landmark try-on here, but I guess it should probably now be converted into XMP data instead: https://github.com/KhronosGroup/glTF/pull/2171 )

[bhouston]

So @donmccurdy how do we go about updating the 3Dmodel XMP data to properly define surface AR - e.g. remove face - and then add additional face landmarks as a secondary type of AR? And then of course there is deformable face layering and hand try on. I guess PRs on the XMP repository? What forum are these discussed?

[donmccurdy]

Sorry, I don't know the answers to that. My guess would be that this repository is still the right place to discuss anchoring-related additions to the XMP namespace. Hopefully someone else can comment here. :)