KHR_animation_pointer

[UX3D-nopper]

[UX3D-nopper]

@lexaknyazev If accepted, squash and merge it.

[rsahlin]

Please elaborate on why this extension is needed and what usecases it sets out to solve.

If this extension is intended as a 'technology enabler' I think care should be taken to confine it to only the core spec, not making it affect current and future extensions by default.

[hybridherbst]

Uploaded some first work-in-progress sample models + videos as a Draft PR:

• https://github.com/KhronosGroup/glTF-Sample-Models/pull/353

Can be viewed in this three.js fork:

• https://needle.tools/three.js/examples/?q=loader_mu#webgl_loader_multiple

[hybridherbst]

Question that came up while implementing export for this extension:

When a material extension property (such as emissiveStrength) is animated,

• it seems obvious that KHR_materials_emissive_strength needs to be added to extensionsUsed on the glTF root.
• does it also need to be added to the extensions of the material itself?
• what should implementations do when a material doesn't specify the extension, but its values are animated? My guess would be that most implementations would then not properly animate those properties (as e.g. the shader may not be the right one at that point in time). Should we make a suggestion here?

[UX3D-nopper]

Question that came up while implementing export for this extension:

When a material extension property (such as emissiveStrength) is animated,

• it seems obvious that KHR_materials_emissive_strength needs to be added to extensionsUsed on the glTF root. Yes, correct.
• does it also need to be added to the extensions of the material itself? Yes, and this independent of the animation pointer. All glTF extensions behave like this.
• what should implementations do when a material doesn't specify the extension, but its values are animated? My guess would be that most implementations would then not properly animate those properties (as e.g. the shader may not be the right one at that point in time). Should we make a suggestion here? This is an invalid glTF. You are accessing a value with the JSON pointer of an extension, which does not exist in the glTF.

[lexaknyazev]

This is an invalid glTF. You are accessing a value with the JSON pointer of an extension, which does not exist in the glTF.

This is a bit more subtle. Many glTF properties are optional with spec-defined default values, so they are always implicitly defined in the asset even if omitted from JSON. Nevertheless, extension objects must be present to be animated.

[UX3D-nopper]

This is an invalid glTF. You are accessing a value with the JSON pointer of an extension, which does not exist in the glTF.

This is a bit more subtle. Many glTF properties are optional with spec-defined default values, so they are always implicitly defined in the asset even if omitted from JSON. Nevertheless, extension objects must be present to be animated.

If the default value is not present, this is okay (This would also conflict with the current validator). But if the whole extension in the material is not present, this is an invalid glTF.

[hybridherbst]

~~That seems a bit ambiguous. If that's the case, it should be stated somewhere in the spec here that a JSON pointer can target the (implicit) default value of an element but not an (implicit) extension of an element.~~

~~And asked differently: does the existance of a JSON pointer imply that the default value MUST be present to have a valid pointer?~~

The relevant section of the JSON pointer spec (if that is what applies here) treats this as error condition but notes that implementations may try to recover from missing values by inserting default ones (no differentiation between "higher-order objects" and "leaf values" here):

• https://datatracker.ietf.org/doc/html/rfc6901#section-7

some applications might stop pointer processing upon an error, while others may attempt to recover from missing values by inserting default ones.

I'm fine with defining it this way – but it means that this PR should specify that the pointer value is not a pure JSON pointer. In contrast to RFC6901, it is valid to target implicit default values of an object.

Also it means that the Validator probably needs to differentiate between "implicit default value is targeted" and "implicit object is targeted". Does that sound correct @lexaknyazev?

[UX3D-nopper]

It is there:

It is valid, to set the pointer to a property, which is not stored in the glTF. This is the case, when the default value is omitted. However, the parent object and/or extension must exist.

[hybridherbst]

Updated my post with an explicit reference to the JSON pointer RFC – I think this could be made more clear (that there's a difference to RFC6901 in this proposal)

[UX3D-nopper]

In general, in any extension or whenever we use a JSON Pointer: The original glTF should not be altered and/or in some way generate data.

Furthermore, with the current animation system, one is targeting the node transforms even these are not stored in the glTF. That is alllowed, as the default values should not be present.

If someone uses the schema, the values are defined, but not present.

[issacclee]

Hi Guys,

Just out of curious, is there anything particular that has stopped this extension from getting ratified? It is flagged as ready for testing, is there anything the community can do to help testing this extension? Such as produce more gltf example model with this extension?

[bhouston]

Let's move this forward! We at glTF Interactivity group, we think this is a great extension.

[bhouston]

There appears to be implementations for Three.js, Babylon.js, and also Unity's glTFast (PR linked above.)

[donmccurdy]

I would request that we not consider three.js a vendor implementation quite yet — Needle Tools has implemented this extension as an addon for three.js, and it's great to see that work, but I'm not sure the three.js maintainers have arrived at an opinion yet. Complexity of implementation is a major concern.

[bghgary]

I will also note that the Babylon.js implementation has two unresolved issues for this extension.

1. We don't support dynamic updates of alpha cutoff values without recompiling shaders. It is possible to implement without recompiling shaders, but no one has ever asked for this and thus we are not sure it's useful.
2. Orthographic camera animations are not working. This could just be a bug in the implementation.

[bhouston]

@donmccurdy wrote:

I would request that we not consider three.js a vendor implementation quite yet — Needle Tools has implemented this extension as an addon for three.js, and it's great to see that work, but I'm not sure the three.js maintainers have arrived at an opinion yet. Complexity of implementation is a major concern.

What is really nice about this extension is that it is a universal interface to modifying a 3D model/scene that works across engines. This enables one to write behavior/logic once and have it work across different engines. It is clearly the future, although I do not know how strange the Needle Tools implementation was, maybe it is just that implementation that is a problem.

I think the next generation of glTF extensions are making behavior portable. This extension is the foundation for portable behavior.

[Hackn0214]

We are developing a 3dsmax version of the glTF import/export plugin.(It is available for download from the Project Explorer page) We are implementing the KHR_animation_pointer extension and are using AnimateAllTheThings.gltf and gltf-viewer-86cef.web.app to check the functionality. Are there any other tools that would be useful to check the functionality? Thanks,

[bghgary]

Are there any other tools that would be useful to check the functionality?

You can use the Babylon sandbox to check also. There are two issues I mentioned earlier.

[Hackn0214]

Thank you, All. This extension has not been ratified yet, but it would be nice to know the status of support for the various tools and libraries.

[issacclee]

Thank you, All. This extension has not been ratified yet, but it would be nice to know the status of support for the various tools and libraries.

I think the most comprehensive implementation of this extension belongs to the NeedleTool

[BeRo1985]

Are there any other tools that would be useful to check the functionality?

You can use the Babylon sandbox to check also. There are two issues I mentioned earlier.

The GLTF support in my PasVulkan project do also support KHR_animation_pointer, see https://www.youtube.com/watch?v=yLIyjX81CB4, so if you do want a precompiled Windows or Linux binary of the PasVulkan gltfviewer example project, then just contact me, then I'll give you some binaries.

Edit:

My PasVulkan GLTF support implementation has also a pretty complete implementation of this KHR_animation_pointer extension, and not only NeedleTool.

[mikeskydev]

What is needed to move this forward? Are we being held up because it's not implemented in something freely available like Blender for authoring purposes? Unreal? More examples needed?

My company is looking to use this extension in production for 3D commerce, hence my work in adding it to glTFast. It would be great to get this ratified to encourage more tools to adopt it.

I already think there's a decent amount of implementations for a pre-ratified extension. 2 implementations for Unity (Covering import and export), three.js (not in main, but at least as a plugin), Babylon, Gestaltor, 3dsMax, various other independent implementations. glTF-Transform is awaiting ratification before implementing.

[Hackn0214]

The 3dsMa version of the import/export plugin has also finished implementing KHR_animation_pointer and will be released once Khronos announces ratification of this extension. We too would like to know the reason for the lack of progress in releasing this feature.

[aaronfranke]

In my opinion, ratification should not be a requirement before implementation. I have interest in implementing KHR_animation_pointer in Godot Engine, and while I haven't done it yet, this is just due to other priorities, not the lack of ratification. When I do get around to it I'd be happy to provide detailed feedback on the spec (to be clear I'm not at all saying that ratification should be delayed for my or other people's feedback, Khronos should proceed how they see fit).

EDIT: That said, did a quick review, there are many problems. This is clearly in need of more review before merging. To everyone reading, I'd like to give a quick call to action: if you want to see something ratified, please review it yourself first and check if it's worthy of being ratified before asking to have it ratified.

[hybridherbst]

@aaronfranke read support for this extension has been implemented in at least glTFast, Babylon, three.js, PasVulkan so far. Write support is in UnityGltf and I believe also in a PR for Blender 4. 3ds max too but not released yet iirc. Thanks for the additional comments!

Most implementations are based on the test models I provided at the top of this PR, you may have missed that since GitHub folds earlier comments:

• https://github.com/KhronosGroup/glTF/pull/2147#issuecomment-1132251826
• https://github.com/KhronosGroup/glTF-Sample-Models/pull/353

I'm happy to create more test models if needed.

[donmccurdy]

While the glTF core specification does define morph target animation, it's unfortunately restricted to animating all or none of the available morph targets. Implications:

1. In a mesh with many morph targets, animation intended to target just one of them must store a large sparse accessor.
2. While one morph target animation plays, nothing else can change the morph target weights.

This issue has come up in three.js discussion threads a few times. My understanding from this section of the proposed KHR_animation_pointer language ...

For clarification, it is not allowed to animate a single element in an array

... would be that KHR_animation_pointer does not lift the restriction. Is that something we should reconsider?

I have been tempted to change the three.js implementation, scanning the animation channel output for morph weights that are 0 throughout playback and removing them from the animation track. But that's a potentially costly scan, and involves some guesswork as to the intent of the model's author.

[lexaknyazev]

This extension is being refactored to decouple specific properties from the general design of animating their values using JSON pointers.

The updated specification for accessing glTF properties at runtime could be found at

• https://github.com/KhronosGroup/glTF/blob/main/specification/2.0/ObjectModel.adoc

[hybridherbst]

@lexaknyazev you mean the specification which properties can be animated now move into that document, and potentially into the respective extensions? Then I think the right link would be: https://github.com/KhronosGroup/glTF/blob/main/specification/2.0/ObjectModel.adoc#5-extension-pointers

I think that document may be missing information regarding how extensions not listed there and future extensions are supposed to interact with pointers. Where would I comment on that?

[lexaknyazev]

That document will be the centralized place for listing standard pointers to runtime properties; it will be regularly updated with future extensions.

The separation is needed because the pointer syntax and semantics are going to be used not only by keyframed animations.