Is there a simple example of using JglTF-model

[Igrium]

[Igrium]

I'm trying to understand how to load a gltf file, go to a frame of the animation, and then extract the vertex data with skinning applied. How do you actually do this? The best reference I can find is gltf-viewer, but that's buried under like 10 layers of abstraction.

[javagl]

It's not that simple.

The skinned vertex positions never exist on the CPU. They only exist on the GPU. They are computed by the vertex shader during rendering. The vertex shader receives a bunch of vertices at their original positions, and a bunch of matrices (namely, the joint matrices) from which the skinning matrix is computed. This skinning matrix will determine the final position of the vertices during rendering.

There currently is no mechanism for directly selecting an animation time. This could be exposed in one form or the other - roughly like this

        List<Animation> animations = 
            GltfAnimations.createModelAnimations(
                gltfModel.getAnimationModels());

        float[] rotation = gltfModel.getNodeModels().get(2).getRotation();
        System.out.println("Before: " + Arrays.toString(rotation));
        
        // XXX The 'update' function is private, but could be made public
        animations.get(0).update(1.0f);
        
        System.out.println("After : " + Arrays.toString(rotation));

This would allow you to access the 'rotation' of the node at a certain animation time.

The non-skinned vertex positions could be accessed with something like

        MeshModel meshModel = gltfModel.getMeshModels().get(0);
        MeshPrimitiveModel meshPrimitiveModel = meshModel.getMeshPrimitiveModels().get(0);
        AccessorModel accessorModel = meshPrimitiveModel.getAttributes().get("POSITION");
        AccessorData accessorData = accessorModel.getAccessorData();
        System.out.println(AccessorDatas.createString(accessorData, 1));

(just a sketch with some fixed indices here).

But then, you'd still have to combine this information, and compute the joint matrices, the skinning matrix, and the skinned vertex positions manually, on the CPU.

There are not really many "layers of abstraction" in the jgltf-viewer, even though I admit that class names like UniformGetterFactory may look a bit obscure when trying to understand the computations that are actually going on. For example, the core for computing the joint matrix is at https://github.com/javagl/JglTF/blob/a016307611a5d6360969e54f5379badee46d03da/jgltf-viewer/src/main/java/de/javagl/jgltf/viewer/UniformGetterFactory.java#L441 , and iff this was not supposed to provide the matrix as a uniform value for a vertex shader (together with all the other uniform values that the shader needs), and iff this was not supposed to handle glTF 1.0 models as well, then one could probably boil it down to a simpler sequence of commands, as in the pseudocode

// Compute the joint matrices
Node nodeWithSkinnedMesh = ...;
Matrix jointMatrices[] = ...;
for (each jointNode) {
    // The global transforms will depend on the animated state
    // of the node TRS properties:
    jointMatrices[j] = globalTransformOf(nodeWithSkinnedMesh).invert();
    jointMatrices[j].multiply(globalTransformOf(jointNode));
    jointMatrices[j].multiply(inverseBindMatrix);
}

// Compute the skin matrix
MeshPrimitive skinnedMeshPrimitive = ...;
Attribute joints = getJoints(skinnedMeshPrimitive);
Attribute weights = getWeights(skinnedMeshPrimitive);
Matrix skinMatrix = computeSkinMatrix(jointMatrices, joints, weights);

// Compute the skinned vertex positions:
Attribute positions = getPositions(skinnedMeshPrimitive);
Attribute skinnedPositions = multiply(skinMatrix, positions);

But this is not something that is intended to be "exposed" by JglTF in any form.

---

That said, and knowing that this may be unsatisfactory: You could probably try opening the glTF file in something like Blender, move the animation to the desired point, and then export the current geometry, as a single frame (assuming that this is somehow possible in Blender...).

It might even be possible to automate that, with reasonable effort, with a small Blender command-line python script....

[Igrium]

Thanks! I was just using the animation as an example. What I'm trying to do is import a glTF file into an existing engine (Minecraft) and given the structure of the rendering engine, I need to arbitrarily set the "current frame" of an animation.

Edit: It's also difficult to integrate into a system with a global time keeper when the only calls to animation update methods are based on system time. For example, what if the user slows the game speed down?

[javagl]

I don't have much knowledge about the inner workings of Minecraft, or where exactly JglTF comes into play here.

This specifically refers to the rendering capabilities of Minecraft. From your description, it sounds like the goal would be to have something like this (ridiculously-high-level) pseudocode

Magic magic = createMagic(gltfModel);
magic.setAnimationTime(12.34);
MinecraftRenderData minecraftRenderData = magic.getAllTheData();
mincraftRenderer.render(minecraftRenderData);

with magic.getAllTheData() essentially returning the geometry from the glTF asset, as-it-is at the given animation time, so that it can be passed to Minecraft as "plain geometry" that can directly be rendered. With a better idea about what such a MinecraftRenderData could be, one could think about which information of a glTF model could be extracted and brought into the required shape with reasonable effort. But things like skinning or morphing will certainly not be supported by Minecraft itself, and these are some of the more sophisticated features of glTF - so there probably will be some limit of what can be supported, unless someone adds full-fledged glTF support in the core of the Minecraft rendering engine...

Edit: It's also difficult to integrate into a system with a global time keeper when the only calls to animation update methods are based on system time. For example, what if the user slows the game speed down?

The question of how the animation time advances is solely controlled by the rendering engine. glTF itself does not have a concept for that, beyond the 'animation time' being supposed to be 'seconds'. Assuming that your comment referred to the AnimationRunner : If someone wanted a different time progression, then this would either require extensions to that class that allow controlling the time progression, or a completely custom implementation of such an animation runner. (The API that may be required for something like this may not be public - but I wouldn't make this public unless it is well thought through and useful...)

An aside: The jgltf-viewer was an early (and probably overly ambitous) attempt to create a glTF renderer in Java, for JOGL and LWJGL. This was fine for glTF 1.0, but but I never got to the point where PBR from glTF 2.0 was implemented properly. Creating a PRB renderer from scratch is ... a bit more involved. The only viable path for me would be to essentially port the https://github.com/KhronosGroup/glTF-Sample-Viewer to Java, but even that would require much more time than I could resonably allocate.

"Pull requests are welcome", as they say...

[Igrium]

If someone wanted a different time progression, then this would either require extensions to that class that allow controlling the time progression

That's exactly what I'm getting at. For anything more complex than a simple viewer, people need control over exactly which frame of the animation is currently being displayed. Or at least the ability to manipulate the playback.

[javagl]

I'd also like a full-fledged, feature-complete, PBR-capable glTF viewer, with debugging capabilities like https://sandbox.babylonjs.com/ , maybe even editing capabilities like https://gestaltor.io/ , and with details like a configurable animation playback speed, and a nice underlying API for people to integrate it into their projects, written in plain Java.

But JglTF is a one-man-show, and only one of (too) many projects that I'm solely maintaining in my spare time, so there's not toooo much to expect here...

[javagl]

No further actionable items here.