glTF
glTF copied to clipboard
KhronosGroup
glTF roadmap - what would you like to see next in glTF?
Hi all - please chime in with any and all feedback to help drive the direction of glTF beyond 2.0. Even simple +1/-1's for topics are appreciated.
How much should we focus on building out the software ecosystem vs. moving forward the spec?
Should new spec features come in as extensions first, direct spec updates, or some combination? What mix?
Ecosystem Tools
What glTF software do we have the most immediate need for?
- glTF 2.0 in COLLADA2GLTF
- One Unity exporter. glTF-Tools-for-Unity, Unity-glTF-Exporter, UnityGLTF
- Unreal
- Autodesk
- Blender import
Conformance
- How do we ensure a robust interoperable ecosystem?
- How rigid and formal should conformance testing be?
- What software tools are needed beyond the glTF Validator?
Infrastructure
- How important is it for the validator to have a JSON schema for validation report, and a command-line and/or web service for integration?
- JSON Schema 6. #929
- Spec contributor's guide: glossary, terms. #1004
- glTF Samples CI
- Validation works already, we could also implement auto-generation of ToC, GIF-previews https://github.com/KhronosGroup/glTF-Sample-Models/issues/26
Learning Material
- What additional tutorials, sample models, and other learning material are needed?
Potential spec updates / extensions
- LOD - BabylonJS prototype extension #1045
- Progressive Geometry Streaming - Fraunhofer POP Buffer, WEB3D_streaming
- Multi-vendor texture compression
- PBR Next
- E.g., transparency/refraction (#1027), multi-pass (#163) and clear coat (#1040)
- API-specific features (they’re interconnected):
- Textures: cube maps, formats, etc. #835, #620, #640, #728
- Enhanced ES 3.0 / WebGL 2.0 support #832
- Enhanced Vulkan support #663
- Point Clouds
- JSON reference other JSON #37
- Bounding Volumes / Collision Volumes #507
- Animation Compression #731
Really looking forward to continuing to move the field forward! Can't wait for your input!
The 2.0 specification is solid. I'd much prefer to see the software ecosystem building out than to see a glTF 3.0 specification.
Agreed with @alteous that the glTF 2.0 spec is feeling pretty good at the moment. It addressed some real issues that emerged with the first spec in a very direct and useful way. I'd prefer to see new features emerge as extensions at this point rather than putting too much effort into a 3.0.
I would like to see an emphasis on tool ecosystem improvements. Especially in terms of exporter and other pipeline support. At the moment finding tools that meet a variety of needs is a bit of a coin toss even with some of the Khronos-backed tools. (I recently pushed a small change to the Blender exporter for this exact reason.) One nice thing to have available would be a collection of sample export targets similar to the fantastic set of sample models: A bunch of Collada, OBJ, FBX, etc files that all have different properties which tools developers can use to sanity check their exports. (Did the model with vertex colors come across right? Did the fact that this one has no UVs screw me up?)
Another thing that I'd like to see discussed is a way to verify/sign that a glTF file has specific properties that make it safe to use in a security-sensitive environment when loaded from a third party. (For example: Oculus mentioned using glTF files as favicons in a VR browser.) It's easy in that type of environment to say "We don't support extensions, animations, etc." but it's harder to usefully say "We won't attempt to render objects over 500k triangles" or "We won't load models that are collectively over 10MB" or "We don't support models with more than 100 different materials" which something like a browser would definitely want to do. The idea being that it could be relatively simple for a malicious source to provide a model that contained a few million individual meshes, each made up of a single, transparent, overlapping triangle that each have their own material each with a collection of unique 4K textures which is intentionally aimed at tanking the client's performance. Ideally a model could digitally verify before hand that it does, indeed, contain only 3 meshes with a mere 8k triangles with two materials between them that only use a couple 512 textures, so there's not a GPU in the world that should struggle with that.
That's a Hard Problem™️ , and I can't pretend to even begin to have the answers for it, but if solved it could make glTF a useful format for a massive range of applications that would otherwise balk at accepting external meshes.
+1 on "Progressive Geometry Streaming - Fraunhofer POP Buffer, WEB3D_streaming"
@toji and @alteous took the words out of my mouth. glTF seems like it's in a good place with its featureset, but it's been very difficult to find a glTF asset pipeline that works well. Solid tools for getting glTF data into and out of all popular content creation tools would be a great next step. Some of the older tools like obj2gltf and FBX-glTF desperately need updating to the new specs to be usable (obj2gltf seems like it's making good progress on 2.0 lately, but FBX-glTF hasn't been updated in a year and has dependencies on projects which have been renamed and absorbed into other projects).
Very happy to see blender import support is on the list already - export without import has left me feeling a bit hamstrung sometimes, so this is something I'm definitely looking forward to.
For software ecosystem I would like to see official UE4 import and blender export. glTF is a good candidate to replace fbx, moreover it's open, so it should have better support in all pipelines/software data exchange.
For conformance, I'd like to see more sample models, covering the whole spec. I already noticed a few missing things while developing my little viewer:
- indices with component types UNSIGNED_BYTE and UNSIGNED_SHORT
- color attributes, with all combinations of allowed types (only SmilingFace has one, but it seems to be black)
- more than 1 texture coordinate set + a material that uses it
Definitely agree that the most urgent issue is improving the ecosystem so that we can all easily make use of what GLTF offers today, which is still a bit too hard to do.
Down the line though, I'd give a big +1 to streaming capabilities, it would be a massive improvement for serving GLTFs on the Web
Hi, I recently switched a company internal project from FBX to glTF (2.0). Here are a few tidbits that I would like to see:
- a C or C++ reference implementation for creating/modifying assets (especially one that comes with few to no external dependencies)
- an extension for custom shaders as pre-compiled SPIRV binary buffers
- an extension for rigid body physical parameters for nodes (weight, joint rigidness, ...)
- an extension for soft body physical parameters (deformable vertex hull, ...)
- a compression mechanism for buffers (could be zip or lz4)
Although I agree that the extensions listed above could be easily implemented as custom extensions, I would like to see efforts to agree on a standard for physical data.
Best regards.
This is really insightful feedback, thanks everyone, keep it coming!
To summarize so far:
- More focus on ecosystem than future spec/extensions (not to say no effort on spec/extensions)
- Sample models
- Many more to test runtime engines, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319113239
- A new set of COLLADA, OBJ, FBX, etc. models to test converter/exporters, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-318931441
- Ecosystem
- FBX to glTF
- Blender export 1.0
- Blender import
- UE4 import
- C++ sample, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319255751
- Spec/extensions
- Voxel
- Security: https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-318931441
- Physics: https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319255751
- Pre-compiled SPIRV
- Streaming
a compression mechanism for buffers (could be zip or lz4)
@KageKirin check out the draft Draco extension. The bitstream spec has reached 1.0 so we expect the glTF extension to move forward quickly, #874
@jbaicoianu obj2gltf 2.0 support should be ready any day now, would love your help testing the 2.0 branch: https://github.com/AnalyticalGraphicsInc/obj2gltf/pull/67
Please support multiple animations (#1052). Only supporting one makes it pretty limited (and a hassle) for exporting to a game engine. All modern DCCs (Blender, Maya, Max, etc) support animation clips, and all game engines support multiple animations, so this would make the workflow much better.
I've posted both a glTF 2.0 exporter and now a glTF 2.0 importer for Sketchup on extensions.warehouse.com, but both are currently pending approval - not sure how long this process will take.
I'd like to see the specification document improved. I think the minimum and maximum values in the accessors were a difficulty for me, especially as they are conversions from float to string and I was plagued by rounding errors making the validations fail. There seems to be nothing in the document to explain why they are required, other than "there are cases when minimum and maximum must be defined". In the end my minimum and maximum values for normalized values looked like "min":[-1.0001,-1.0001,-1.0001],"max":[1.0001,1.0001,1.0001]
Decomposable matrices were another issue, and a matrix which I thought was decomposable (a matrix built from translate, scale & rotate) would still fail validation - again I don't know if it is because of rounding issues converting from float to string - but it would be nice if the spec included the code to decompose the matrix and what the expected precision and validation requirement would be. In the end I had to give up and just pre-multiply all the geometry by the transformations first, so exporting with identity matrix only for my version 1.0.0 exporter.
Other parts of the specification could have a bit more of a tidy up, e.g. "When nor sparse, neither bufferView is defined..."
Please support multiple animations (#1052).
I've posted more details on #1052, but this is already supported — our examples do not make it obvious. Definitely +1 from me on changing the example models to not use different animations for each node, and adding some examples that have multiple distinct animations.
What is the status of hdr formats? Openexr would be standard, but I'm open to ideas.
Openexr supports both 16 bit float and 32 bit float which corresponds to OES_texture_float; OES_texture_half_float.
Actually in general the missing bit to me is more about being able to add extensions/plugins. I strongly agree that 2.0 is already very strong and doing a lot, and going too deep may start to make it too specific (let's not make it grow into an FBX-style format).
So my main suggestion for extension would be to introduce some type of standard plugin solution that could allow custom extensions to be added and embedded inside gltf data.
As a simple example the rigging part is pretty standard today to map what you can do in Unity, but if you are in need deeper notion of constraints or deformers then you're stuck with only skinning/shapes.
The problem being that plugins mean evaluation mechanism during playback, most often seen as a complex problem. I actually think it's pretty simple if we could have a notion of scripted based behavior with embedded js or something like this (call it scripted operators, unity behaviors, or whatever). And avoid the big language debate by making this js-only since it's targeted mostly for web.
And btw, neither FBX now dot.xsi / KL / ... really tackled this right since they are all biased as companies, we have the luxury to make it right, and this could grow a huge community of developer IMO.
@erwanmaigret
I don't think adding a scripting language to a model format is a good idea, especially JavaScript. What I would rather want to see (I haven't really read spec, so this might already be supported) is arbitrary data. Meaning you could embed anything you want to the file, possibly in base64 encoded binary.
@Noxime: That already exists. Buffers and BufferViews in glTF are references to completely generic binary blobs which can contain anything you want in any format you want. If there's data that's unique to your use case you can easily create an object in a relevant extras property somewhere that points to a BufferView and provides whatever metadata is necessary to interpret it.
@noxime
Both javascript and arbitrary data are bad idea imo because this way you're are creating a vector of possible virus/exploit specially on pour coded library. With script you basically allow execution of arbitrary code, and arbitrary data may cause exploit by using particular string(something like happened to WhatsApp some time ago).
For conformance: Also, I don't like much the idea of "extensions": Having an extension mean that an implementer may or not support it. This mean that a software may implement it and another one no. In case one want to pass data from this software to the other it virtually lose information. Instead a non extension spec, must be implemented to be compliant with the implemented version. There is also the problem of bloat the format with less used or exotic stuff..
More information on glTF's extension system, and why we have it, here. Extensions might specify extra data, intended to be used in a particular way by the runtime, but will not include scripts implementing that behavior.
Implementation of an extension's behavior may be handled by extending the loader for a specific engine, rather than as part of the glTF format itself. For example, we (three.js) are hoping to allow users to write plugins for THREE.GLTF2Loader without having to modify three.js directly (https://github.com/mrdoob/three.js/issues/11682), so that you can experiment on extensions as needed.
Also, I don't like much the idea of "extensions": Having an extension mean that an implementer may or not support it.
The intention here is, features that can and should be supported by all implementers will be part of the core specification, not extensions. As such, content authors who want their models to run everywhere should stick to the core spec.
However, we don't want to limit glTF so much that it only supports features possible on low-end mobile devices. For more expensive materials (like PBR specular-glossiness, or SSS) or WebGL 2.0 features, we hope to use extensions to enable experimentation, and then these features can be brought into the core spec if/when the time is right.
I totally agree bringing some language in the game is far from ideal. But then I don't see any other options unless we start to get into predefined constraint+deformer notions.
Both could work in the end. I am afraid that if we don't do either, then the format will be limited to embedding only skinning/shapes/animation forever and side solutions will start to be created.
Let's just consider a very simple aiming constraint, which is very commonly used today in about any rig with a character. How could we make it so it's part of the data part?
Or maybe this just does not fit into the low-level gltf format but then we should consider specing out an official solution as a standard gltf behavior manager?
I disagree that glTF needs an embedded behavior system. At the end of the day, glTF is specifically a data transmission format. It's the jpeg of 3d, not the javascript. I think it would be useful to be able to serialize and transmit "entities" instead of "models", but trying to shoehorn that into glTF is a mistake. That deserves its own spec IMO.
@erwanmaigret: Things like aiming constraints sound highly use-case (and probably engine) specific. That's what the extras attributes are for. Certainly there's nothing preventing you from putting a full scripting language into an extras attribute if you wish, and your content pipeline can be tuned to use that, but everyone else loading the file will only use the parts of it that are standardized. That sounds problematic, but in reality there's very little chance that something as specific as an aiming constraint will map cleanly between environments anyway.
@toji Make sense I'll write this as a rigging/behavior engine riding on top of gltf then in a separate repo.
(Minor summary update; italics indicates updates from https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-320047171).
- More focus on ecosystem than future spec/extensions (not to say no effort on spec/extensions)
- Sample models
- Many more to test runtime engines, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319113239
- A new set of COLLADA, OBJ, FBX, etc. models to test converter/exporters, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-318931441
- Ecosystem
- FBX to glTF
- Blender export 1.0
- Blender import
- UE4 import
- C++ sample, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319255751
- Spec/extensions
- Voxel
- Security: https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-318931441
- Physics: https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-319255751
- Pre-compiled SPIRV
- Streaming
- Spec doc improvements, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-320078379
- hdr, https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-320154633
Personally, I'd like to see focus on complexity management e.g. supporting levels-of-detail (and streaming), being able to support structured breakdown (json file referencing other json file(s) - #37 ) and finishing up the alternate material support e.g. a simplified blinn/phong model.
For the sake of simplicity and ease of adoption, I hope that the scene hierarchy remains a strict tree. I say this because the nodes and heirarchy section of the specification mentions that this restriction may be lifted in the future.
I would assume that if this is going to be truly platform neutral, there is going to a cleanup of some of the GL-specific enums e.g. componentType : 5126 (GL_FLOAT).
In case you care, https://github.com/xelatihy/yocto-gl has a C++ loader/writer and example code for both a small viewer and OBJ2GLTF. All C++ and tested on all models from the glTF 2.0 spec.
As for the new spec, I would suggest
- external mesh file
- reduce complexity of the format
- added extensions for subdivision surfaces, splines (see OpenSudDiv)
- Disney PBR BRDFs
Let me comment a bit, and please consider the fact that I am mostly interested in C++ and fast execution speed. glTF might be a great fit for JS and WebGL, topics on which I am very ignorant about. 1, 3 and 4 can be done with extensions, but 2 cannot. But extensions are not that useful since tooling won't really follow them.
glTF has a ton of indirection that I would love to see removed. For example a. mesh -> shape -> accessor -> bufferview -> buffer -> data b. material -> metallic-roughness -> texture_info -> texture -> image -> data or buffer All of these slow down native code and are sources of errors for data files on the web.
Also, for our projects, it is very problematic that buffers are not typed. When we work on asset we can easily work on images using PNG. PNGs can then be shared with different glTF without requiring modification to the JSON data.
For meshes, we cannot do that since the spec forces us to change all glTFs that reference a buffer. But if I just want to add data to a mesh, like tesselate, I should not have to change all scenes that they are referenced into. Also, we can easily work on image files in parallel, but cannot do so on meshes again since all changes needed to be manually reflected in glTF. This makes a lot of asset processing way more complex than it should be and makes meshes not behave the same as images.