glTF
glTF copied to clipboard
KhronosGroup
FB_ngon_encoding extension.
We're in a situation where we want to do subdiv rendering on clients, where we want a quad-based mesh, and yet take advantage of all the backend infrastructure we've built around glTF. We came up with this scheme to retain general ngon mesh structure in glTF-valid triangle meshes.
We put this together for internal backend->client delivery & rendering, and there's no current plan to offer explicitly user-friendly download of assets that use it. Then again, they do get sent over the wire; folks will surely intercept & inspect them, and it feels like a goodness for any extended glTF that makes it into the world to be documented somewhere.
I'll leave it to y'all to decide if it makes sense in this repository. If not, I will probably drop it into the FBX2glTF repo, so that tool's output is at least documented.
@zellski thank you for proposing this.
I want to provide some thoughts on quads/subdivision in general in glTF. glTF was started in 2012 for efficient runtime and transmission. In the early days of glTF if you asked me about adding subdivision, I would have said that (1) that would be slow to compute at runtime, and (2) it would make the spec more complicated to implement and increase the barrier to adoption. I might have been right in the context of 2012 when there was less client-side processing power and it was pre wide-spread glTF adoption.
Today I think the definition of efficient runtime and transmission may be different and subdivision surfaces may fit the spirit perfectly. There is more client-side compute available for subdiv, the storage and bandwidth savings are significant, and the USDZ to glTF conversion pipeline will be cleaner. I also suspect the implementation effort is reasonable and still in the spirit of glTF, especially now that so many glTF runtimes exist and that open-source libraries can make integrating subdiv straightforward.
So whether it is this exact extension or a variation of it, I think this is a great direction for the glTF community to explore.
This is one of two plausible ways we brainstormed to provide information sufficient to reconstruct a quad / ngon tessellation from a vanilla-compatible triangulation.
The virtue of this proposal is compactness; it adds no size to the asset, but rather takes advantage of the redundancy of order-independence. That is also its downside: with this extension in place, tools are longer at liberty to reorder triangles and vertices as we please. Pipeline tools will have to make potentially annoying exceptions to optimisation steps when they encounter this extension.
The obvious alternative is to let vertices and triangles remain order-independent as per vanilla glTF. In this case, the information needed to reconstruct the original mesh has to be provided explicitly, likely through the use of an accessor. I imagine there's a few different ways to encode the required information, but one way or another it's likely to consist of references to vertices and/or triangles. Given that fact, I'm not sure we gain any real sturdiness compared to the implicit approach above. A pipeline tool will still have to understand the extension in order not to screw up those references with a hamfisted reordering.
I wonder if it's worth adding a flag here, to indicate whether the ngons are intended to be used as-is, or whether they represent a subdivision surface that should be further processed by something like the OpenSubDiv library.
That is also its downside: with this extension in place, tools are longer at liberty to reorder triangles and vertices as we please. Pipeline tools will have to make potentially annoying exceptions to optimisation steps when they encounter this extension.
This actually seems OK to me. In general, an optimization tool probably cannot and should not pass along an extension it doesn't recognize. To give a simple example, any extension could reference accessors, and an optimization that alters accessor order would trivially break the extension. The fact that this extension design allows the optimization tool to read a file with FB_ngon_encoding, optimize what it understands, and output a correct file (without FB_ngon_encoding) is a perfectly acceptable example of fallbacks working correctly.
I wonder if it's worth adding a flag here, to indicate whether the ngons are intended to be used as-is, or whether they represent a subdivision surface that should be further processed...
Not sure I understand – can anything be done with quads as-is, except (a) subdivision or (b) loading into a DCC tool? From my perspective, (a) is the only compelling reason to consider this extension, given the stated goals of glTF.
perfectly acceptable example of fallbacks working correctly.
Agreed. And this is an amazingly elegant solution to the question of quad/ngon storage.
Not sure I understand – can anything be done with quads as-is, except (a) subdivision or (b) loading into a DCC tool? From my perspective, (a) is the only compelling reason to consider this extension, given the stated goals of glTF.
Not all meshes are intended to be subdivided, for example a cube becomes a lumpy ball when subdivision happens. If that's the only reason this extension exists, then subdivision should be in the name of the extension, and clearly stated as the intended use of the quads & ngons.
If this extension is raw quad/ngon storage, and you find a cube of quads stored here, then what did the artist intend the viewer to see? Did the artist want a cube shown, or a ball? Granted, the end user's reasons for sending a non-subdivided cube as quads may be less obvious to the authors of the glTF spec, but we should either enable it or rule it out completely. The README Overview should not say "one common such case" is subdivision, when turning that on or off may have a large impact on the visual result. Subdivision needs to be an intentional choice by the artist, not a client interpretation.
And if we're targeting subdivision exclusively, do we need to include a mechanism for subdivision weights or creases? Is quad/ngon storage enough input to supply to OpenSubDiv? I'm not trying to overwhelm this PR, just trying to think through ways people might use this in the wild.
Concave polygons can not be converted into a valid triangle fan. But I guess this extension is for convex only polygons. I must admit that it is a very elegant solution for convex polygons.
I do think that my extension that adds an edge data array covers both concave and convex polygons as well as edge creases:
To make it concrete one can support subdivision and polygons with one array of edge values stored as signed bytes. A negative value means not a true edge (interior poly edge.) A positive value should be considered a crease weight between 0 and 1 of which there would be 128 possible discrete values.
Just define a deterministic face indices to edge indices ordering.
If this isn't supported you can still load and display the data as a normal glTF. If you do support this you can adjust glTF resolution.
This single linear byte array, which is no larger than 1/3 the size of the face indices array uncompressed, will compress very well using zip or brotli.
My proposal is also fully backwards compatible as well if the extra data is not supported.
Not all meshes are intended to be subdivided, for example a cube becomes a lumpy ball when subdivision happens. If that's the only reason this extension exists, then subdivision should be in the name of the extension, and clearly stated as the intended use of the quads & ngons.
This is a misunderstanding of subdivision. Correct subdivision involves the use of a value per edge that says whether it is a crease or not (or a value in between if you are doing crease weights). One smooths across edges that are not hard creases. Thus you can subdivision a cube and if you have specified it to have hard edges, it increases the polygon count without actually rounding the corners.
A lot of libraries, such as Three.JS have implemented subdivision incorrectly by assuming all edges are not creases.
Be aware that there are a few different crease representations:
- In 3DS Max, it uses face smoothing groups and an edge is considered hard if adjacent faces do not share a smoothing group in common.
- In vertex normal representations, if an interior edge has only one set of vertex normals rather than two sets, then it is smooth. Otherwise it is a hard crease.
- In tools such as Maya that support variable weight creases (which aligns with OpenSubDiv), there is a scalar per edge that specifies how hard it is.
It is important to note that in complex models there may be different edge crease weights for normals and uvs. In those cases I suggest that one assume they are the same, and only if one specifies explicitly different edge crease weights for the other channels that they are different.
The proposal I am making supports all three because the most expressive/general format is the crease weight system. The other representations can be converted to it.
@bhouston Are you referring to another proposal or PR for subdivision? I'm not sure what you're quoting from.
@bhouston Are you referring to another proposal or PR for subdivision? I'm not sure what you're quoting from.
Well, it was just an email on the glTF mailing list -- and that was it -- thus it isn't formalized yet, but I think it is reasonable. Basically you need data per edge. Edge indices can be calculated deterministicly from a set of triangles -- though I am unsure if it is a O( t ) or an O( t log t ) operation where t is the number of triangles.
Once you have an edge indexing scheme, you can add attributes per edge as simple flat arrays. These can be a simple boolean interior edge or not -- this would give you a polygonization. And then more specific to sub-div, you can have edge crease weights for the various channels (normals, uvs, etc.)
My team and I at Apple worked with the OpenSubdiv team to identify the minimum viable set of data to describe subdivision data, you can read our specification here:
https://developer.apple.com/documentation/modelio/mdlsubmeshtopology?language=objc
In addition to edge and vertex crease info, we found it necessary to also encode faces that function as holes. I think the information in our specification for ModelIO is complementary to this proposal.
Indeed; I described the limitation in this PR as "It's obvious that this can be done for convex polygons, e.g. quads, but the true constraint is a bit less strict."
I'm not really competent to evaluate @bhouston's solution, but I'm happy to assume at face value that it solves useful cases that this PR doesn't—ours was really hand-crafted for one specific purpose. If so, perhaps we should base a future EXT version on the more general solution, and not promote this vendor one any further.
That said, we're already fairly committed to this one internally, so if users go out of their way to intercept our server-to-client delivery GLBs, those may well require FB_ngon_encoding. If we ever added a 'download' button, we'd probably want to mark those up with an EXT.
Not all meshes are intended to be subdivided, [...]
This is a misunderstanding of subdivision. Correct subdivision involves the use of a value per edge [...]
Actually I'm trying to point out uses outside of subdivision entirely, for example BIM data, OpenStreetMap etc. There may be reasons (possibly analytical reasons) to preserve quads rather than triangulate everything. But one wouldn't want to ship edge weights or subdivision implications along with such data.
As a much more concrete example, we now have a PR open to import FB_ngon_encoding data into the official Blender importer, in https://github.com/KhronosGroup/glTF-Blender-IO/pull/622. This import does not add the Blender "Subdivision Surface" mesh modifier when the data comes in. But should it? Should it blindly assume that the mere presence of FB_ngon_encoding implies that the subdiv modifier always gets applied? And why is the import software making such decisions on its own, shouldn't it be informed by some setting in the model?
And to be clear, I'm fine if the answer is "Yes, the mere presence of FB_ngon_encoding means that subdivision itself as a feature should be enabled", but the README needs to say so. Right now the README just says that clients can turn subdivision on or off at their own discretion.
(1)
Should it blindly assume that the mere presence of FB_ngon_encoding implies that the subdiv modifier always gets applied.
The mere existence of polygons do not imply subdivision unless you want to do it incorrectly. That said this is a vendor standard so the bar for rigorousness/correctness is lower as it merely had to meet FB's needs, not be how the rest of the high-end 3D industry does things.
(2)
I would also suggest renaming this to FB_convex_ngon_encoding as it doesn't handle convex nor holes.
(3) The correct solution as Nick (@meshula) points out is to have true polygons and edge crease scalars and even vertex crease scalars.
My suggested backwards compatible alternative of an array of booleans, one per edge, to differentiate interior edges from polygon border edges, handles both concave polygons as well as holes in a backwards compatible fashion.
And then to do correct subdivision, add an identically sized array of scalar to specify edge crease weights to handle the proper subdivision boundaries. To accommodate Nick's suggestion of vertex crease information add another scalar array of the count of vertices -- although in practice I have rarely seen this used, rather I have tended to use a heuristics to derive vertex creases from edge creases.
There may be other ways to do a fully correct solution, my specification may not be optimal, but the currently proposed specification is inadequate for a formal standard which values correctness.
(4) But this is not a standard proposal, rather just a vendor solution, so go with this extension and get it done.
(5) If Khronos is adding support to the blender glTF extension for polygons based on this proposal on the assumption that this is the correct way to do it and it should become a defacto standard, I fear we are jumping the gun, and going with an inadequate solution out of expediency.
Khronos itself isn't adding support, this is an external contribution PR to the open-source importer. It hasn't been merged yet, and I'm concerned that merging it implies more official support than perhaps we would all like at this early stage.
Khronos itself isn't adding support, this is an external contribution PR to the open-source importer. It hasn't been merged yet, and I'm concerned that merging it implies more official support than perhaps we would all like at this early stage.
I'm not looking to jam up or act as a spoiler for Facebook's efforts. Thus maybe just mark it as Facebook Convex polygons or something so it is there but clear that it isn't proper subdivision.
To reiterate with further clarity: we (FB) consider this extension worthwhile mostly as a formalisation of our own internal client/server asset markup. These models are not intended to leak into the world, but nor do we try to hide them. The PR was our attempt to offer websearch-discoverable documentation for if/when people do come across it. I think the esteemed Mr. Mackey is correct that merging this risks undermining a future extension, and quite generally confounding. If not merged here, I will add it to the FBX2glTF repository.
Is there any progress on accepting this Quad GTLF extension?
- Adds no extra filesize
- GLTF viewers display the mesh correctly even without implementing the extension
- This is already implemented in Blender, which is a significant step in advancing glTF authored assets in general
What's preventing this quad support from becoming an accepted extension exactly?
Quads for a 3D runtime format has a LOT of use cases like runtime adaptive subdivision level of detail, and topology analysis by showing the wireframe of a quad mesh.
Sites like sketchfab show wireframe previews on quad meshes, but when you download as GLTF, there is no way to preserve the topology of the model on reupload.
.obj is still used heavily over GLTF because it's one of the few free 3D formats that support quads.
Modern 3D modeling workflows are extremely quad oriented for topology and subdivision. Projects like OpenSubdiv from Pixar literally cannot use GLTF because this format doesn't support quads yet. For the purpose of wider 3D application adoption, GLTF really needs to support quads.
What's preventing this quad support from becoming an accepted extension exactly?
Yeah, at this point I'd propose the PR be merged as-is, a vendor extension. Its limitations are spelled out clearly enough that it shouldn't damage the adoption of a more sophisticated EXT_ version down the line, but that could be a while – and meanwhile, this does provide glTF with quad support.
I advocate it stay as a vendor extension and we merge it as is.
It is clear it does not meet the needs for DCC-to-DCC loss-less quad/polygon transfer (as it doesn't support multi-topologies) like USD does, nor does it meet the needs for fast/easy GPU subdivision surfaces (as it doesn't store vertex or edge creases, nor guarantee quads, nor provide a limit surface fallback) as Alexey's proposed Quad SubD extension does.
This extension is sort of in an in-between space - quads by themselves.
Here is a presentation that puts this extension - the last slide - into context with Alexey's Quad SubD extension and how traditional raytracers utilize SubD on multi-topology meshes: https://docs.google.com/presentation/d/1-55EI56KJQvw-oeGcZUpkz6Qd4NhZgXwJqV4u0SnBzU/edit
I hadn't understood about corner/crease sharpness before. We could also skip this extension entirely, if we have something else more or less ready to go. I just worry that we're all limited on bandwidth and we don't want this to sit idle for a year.
@zellski if you feel this is ready to merge (as a vendor extension for now) could you:
- [ ] Change status to "Complete"
- [ ] Add FB_ngon_encoding to the vendor extension list
... I'd unfortunately need to vote against merging an implementation of the vendor extension in common tools ...
After some discussion, I feel less strongly about this. Perhaps support could be merged to Blender as-is, disabled by default, and with some warning that it may eventually be replaced by an official extension. Thoughts?
@zellski any update on @donmccurdy's comment above? It would be great to merge this. 😄
Apologies for the delay; I hope to make the changes shortly. I quite agree with Don's general summary. Let's make certain there's no confusion – merging this vendor extension does not mean glTF's subdiv needs have been met.
Hi all!
I'm working for Adobe and we'd love to push this extension live! We intend to use it and want to make sure it gets documented properly and fully approved by the Khronos Group before using it. I see that this has been stalled for the last year. I can do the final touches to the PR, it seems we only lack easy changes to make it (judging by @donmccurdy's last comment).
@zellski Does that sounds good to you ? If FB doesn't see value anymore in this ext, we can take over the ext and prefix it ADOBE_ if that's fine to everybody. We'll be able to maintain, document & discuss the ext if need be.
My constraint would be that it has to come from a fork we (Adobe Github org) own (so I'd need to create a new PR, etc...). Would that work for everybody here?
Hi @clems71, apologies for the slow reply here. This extension represents a feature wanted or needed by many (quads and ngons) but does so in a way that was meant to be internal to a single (large) vendor. Thankfully it does so in a brilliantly backwards-compatible way. But it also does not indicate whether subdivision should be enabled or disabled, or what the default edge/crease weight should be, or what individual edge weights should be, etc.
Ideally, in the long run, I think it would be good to repackage this as an EXT (multi-vendor) or even KHR (Khronos ratified) extension, with additional settings in the extension to indicate subdivision enabled/disabled, default weight, and possibly call out a custom vertex attribute or something for individual edge weights.
But as you noticed, this has been on the backburner for over a year. Renewed interest from a major software package could potentially sway the Working Group's priorities back in this direction. I'd love for the Working Group to hear more about how you might add support for this extension and what the effect would be for users. If you can't share that information publicly, you do have the option to share that privately with Khronos, because Adobe is a full Khronos member. Let me know if you're interested to explore that avenue.
@emackey Our plan is to support quads (and more generally polys) on 3D models for our 3D oriented marketplace : Substance Share, and also other products I can't disclose yet. Our internal representation of 3D models is GLB. We like the format and its extensibility and it is also a format supported by all our tools (Substance Painter, Designer, Dimension, ...).
We strive to deliver the best visual quality AND fidelity. That means for 3D models that if someone uploaded a FBX file with quads (usually for tessellation purpose as you guessed it), we store it internally in GLB, and thus need to preserve the quads along the pipeline, so that people know what they get. Wireframe will really reflect the quads. We rely on THREEjs for our viz needs so we'll likely add open-source contribs to support the ext, as well as in FBX2glTF from Facebook, (we started to work on this one already).
At the moment, we are solely interested in static meshes but being able to support animation will likely be needed in 2nd half 2021.
A quick note here to let you know I drafted a PR around this extension in Assimp library here : https://github.com/assimp/assimp/pull/3695
Actually, this extension can also support any star-shaped polygon, even non-convex ones (@bhouston).
Star-shaped polygons can be described by a triangle fan. An issue may arise if two consecutive star-shaped polygons have a unique choice for the central vertex, and it's the same vertex for both. In this case, ngons would have to be reordered, since vertices cannot be reordered in any of those two ngons.
A typical example of a non-convex star-shaped polygon is a concave quad (arrowhead-shaped). If two consecutive arrowheads share the head vertex, they have to be separated.
