Animation Import for RealityKit

[kaustubh-seachange]

Is there any timeline to implement the animation in RealityKit version of project

[warrenm]

There is currently no timeline for animation import for RealityKit, and it's unlikely to happen this year without external support.

[warrenm]

The chief impediment to implementing this feature is RealityKit's lack of support for keyframed animations with arbitrary time values. RealityKit's SampledAnimation type insists on using a shared frame interval for all keyframes, so to import glTF animations into the system, we either need to resample (expensive and error-prone) or come up with a workaround, such as reimagining glTF keyframe animations as sequences of from-to-by animations (expensive and problematic in other ways).

[issacclee]

One way to look at it is to maybe delegate the obligation of "Resampling GLTF animation data" to whatever gltf exporter the user were using. Someday it might just become a standard option on those exporters to export a "flavor" of gltf specifically for using gltf in Realitykit. For this project, we could just go ahead and implement based on the existing restrain of Realitykit's SampledAnimation data class. Maybe throw an error when the importing gltf file does not share the same time frame interval across its animation channels.

[warrenm]

I'm not interested in implementing half-measures that I know won't work with a significant number of existing assets. The practical outcome of your suggestion would be that people would assume animation support for RealityKit "just works," and then complain when it breaks.

Any user of the framework is, of course, welcome to implement whatever hacks they want to in their own forks and clones.

[giln]

Aren't animationResources in RealityKit a sequence of fromToByAnimations ? Why would it be problematic?

[warrenm]

It is possible to generate an AnimationResource from a sequence of animation resources generated from FromToByAnimations, sure. This approach has at least two issues:

1. As far as I can tell, it is not possible to construct a BindTarget that targets a single component of a Transform, such as its rotation, while preserving the existing scale and translation. It's perfectly valid in glTF to have an animation only affect one part of the transform, but if we just blindly use from-to animations that target the transform, we lose information and probably corrupt a small percentage of animations in the wild.

2. AnimationTimingFunction isn't expressive enough to implement the cubic spline interpolation mode, so we'd either need to approximate it or fall back on resampling.

[giln]

Ok, thank you for the detailed answer.