CesiumGS
Loading some glTF/glb animated models causes severe stuttering in Cesium, with the fps dropping close to 0.
What happened?
Loading some glTF/glb animated models causes severe stuttering in Cesium, with the fps dropping close to 0. I have provided the model in the attachment, and it runs fine in desktop software such as Blender and 3D Viewer.
Reproduction steps
...
Sandcastle example
No response
Environment
Browser: CesiumJS Version: Operating System:
Just for completeness or as a baseline: Dragging and dropping the model into https://github.khronos.org/glTF-Validator/ reports 441 (!) errors. None of them seem to be really "critical", and other viewers can display it without problems. And it sounds like CesiumJS can display it, but with low FPS, so these errors might not affect CesiumJS either. But wherever this model is coming from: It should preferably be fixed before using it further. (One can not expect a viewer to display an invalid model)
Hi @netcanol, can you give us a little more info about this model? How was it generated?
It is a model that was apparently exported from SketchFab, from https://sketchfab.com/3d-models/animated-drone-d47d61f8a78d4cce9f985ad7ade3e403
I had another short look at this. The model has several animations. One of the animations shows how that drone is assembled, from scratch. Another one is an "exploded" view. That looks pretty cool. The default animation should be the "Hovering" one.
In CesiumJS, it comes to a grinding halt when pressing "Play" on the timeline.
There's something odd about the structure. The model uses vertex skinning - which certainly does not make sense for a "CAD"-model consisting of rigid parts. Apparently, the "movement" is modeled somehow via a skinning animation. And for some reason (that remains to be investigated), this model ends up in CesiumJS with 77 nodes where each node has 88 joint matrices. They are likely all the same (there is only one skin), but that doesn't prevent CesiumJS from updating them many (probably 77*88) times...
A quick trace where the time is spent:
Thanks @javagl, I'm seeing a lot of errors like this from https://github.khronos.org/glTF-Validator/:
"message": "Joints accessor element at index 110 (component index 2) is used with zero weight but has non-zero value (81).",
Perhaps other renderers recognize the zero weight and skip the joint matrices, whereas Cesium is assuming everything needs to be updated.
I don't know how worthwhile it will be to debug in CesiumJS until we understand if these errors are relevant. @netcanol is there a way we can get a fixed version of this model? Perhaps load it in Blender and re-export it? Thanks in advance for any debugging help you can provide!
I already started up Blender "yesterday" (i.e. earlier this morning 🙂 ), but that alone didn't fix it. Blender also changed the order of the animations, which is a bit unfortunate. So I ran it through some additional script in https://gltf.report/ to remove the animations that are (likely) not required here, and pruned the whole thing. These modifications originally removed the copyright information that was stored in the asset.extras. So I ran this through a small JglTF snippet to manually re-insert the original copyright information. Call me pedantic, but I think that retaining copyright information is important. It's not only a courtesy to the original author and a form of acknowledgement of the tremendous amount of work that was put into this model. It is also important from a legal perspective: The model is published under "CC-BY-4.0" license, and this cannot just be omitted.
The result is attached here:
animated_drone-CLEANED_UP-copyright.zip
But... ... unfortunately, at least for me, neither the original model nor that pruned model are displayed in CesiumJS, at all. I didn't check it closely yesterday - I only started the animation and looked at where the time was spent.
Here is a screenshot of the original model with debug bounding spheres:
As you can see, you see nothing.
So with that "CLEANED_UP" model, the animation runs without bringing CesiumJS to a grinding halt. But ... it's still not displayed, which is ... kinda funny on one level, but sad on all other levels.
So for both the original and the fixed model it remains to be investigated why it doesn't seem to display anything. But even if the displaying issue was resolved, I think that the issue of the "stuttering" with the original model is still valid, and the reason for why CesiumJS is updating matrices like crazy should be investigated as well.
And for some reason (that remains to be investigated), this model ends up in CesiumJS with 77 nodes where each node has 88 joint matrices... updating them many (probably 77*88) times...
Because the joint transforms are unaffected by any transforms on the skinned mesh node, it should only be necessary to visit each joint once per skin, not once per skinned mesh node. This optimization would fail if there were 77 different skins using the same joints, but fortunately there's only one skin here. Possibly that's the hotspot to optimize?
Copyright info disappears after processing likely because it wasn't located in either of the official places: asset.copyright or KHR_xmp_json_ld. I think Sketchfab's implementation predates the XMP extension and was using asset.extras instead, but I haven't seen that approach elsewhere. Of course that should be preserved too... I'm not sure yet why it isn't:
https://github.com/donmccurdy/glTF-Transform/blob/82db25002d692cc327ebb63d454b67840494c5fa/packages/core/src/io/reader.ts#L59
Possibly that's the hotspot to optimize?
Skipping musings about the line between "optimizing" and "fixing a bug", it strongly looks like CesiumJS is duplicating some matrices, ending up with these 77*88 matrices, where one of the factors is only coming from that duplication. It's not unlikely that this was never noticed just because such a structure is unusual (it won't be noticed for the SimpleSkin tutorial model).
Copyright info disappears after processing likely because it wasn't located in either of the official places:
I'm a bit surprised. I would have sworn that glTF-Transform overwrites the copyright with its default. (Random but somewhat "related" crosslink: https://github.com/KhronosGroup/glTF-Sample-Assets/pull/250#issuecomment-3534653882 ). Maybe that depends on the transforms that are applied or how exactly the model is written? Do you know someone who's familiar with glTF-Tra.... wait 😆 Maybe my memory is hazy here - the investigations above had only been a quick, first step, to see whether it's actually a CesiumJS issue (which it appears to be), and whether it's easy to salvage the relevant part of the model (which doesn't seem to be the case...).
I would have sworn that glTF-Transform overwrites the copyright with its default.
Might be thinking of asset.generator! That one is more complicated, since (1) glTF Transform might not be the original generator of the model, or (2) the user might be using glTF Transform as a dependency of a larger application, and want the larger application's name in the generator string.
Agreed though – it's probably a fairly unusual case with the skinning setup here. As long as the parent node's transform isn't affecting the joints, I think it's mainly an optimization.
Yes, sorry, I mixed up generator and copyright here (am a bit distracted right now). And the generator should IMHO indeed be the tool that is "responsible" for the asset in its latest form, so all good.
I haven't figured out why the "simplified" model wasn't rendered in CesiumJS, though. It is valid, and it does render properly in your viewer. There are some warnings "NODE_SKINNED_MESH_NON_ROOT - Node with a skinned mesh is not root. Parent transforms will not affect a skinned mesh" - maybe CesiumJS is somehow not handling that well...? I'd have to check more closely.