Follow-ups for Gaussian Splat SPZ support

[javagl]

I mentioned that in a comment in the Gaussian Splat SPZ Support PR, but it may be worth tracking this here:

---

There seem to be some assumptions about the structure of the glTF of which I'm not sure whether they will always hold: https://github.com/CesiumGS/cesium/blob/8a362a0aa2e038cde8e0a50fd83d60e20b1eb9c1/packages/engine/Source/Scene/GaussianSplat3DTileContent.js#L381 There is not necessarily a matrix property in that node. The first primitive may not be a Gaussian Splat primitive (or the Gaussian Splat primitive may not be the first one). There may also be multiple Gaussian Splat primitives in one glTF.

---

EDIT/Update: The following was addressed via https://github.com/CesiumGS/cesium/pull/12706 (additional test cases may have to be considered):

Modifying the tileset modelMatrix does not seem to have an effect on the gaussian splat primitive. (Or at least not the "right" effect: It seems to affect the position and bounding volumes, but not the orientation).

---

Speaking of orientations: People might say that I'm obsessed with unit cubes. And I wouldn't even concur 😁 : I created a "Unit Cube Splat File". (It's rather a 10x10x10 cube, but unit-y in other ways). And when rendering this in BabylonJS, it looks like this:

Exactly as expected. With the colors and such. 🔴 🟢 🔵

When putting this into a glTF and a tileset and rendering it with CesiumJS, it looks like this:

There may be a million reasons why something is "wrong". For example, the bounding box. This is coming from the tileset JSON, and may therefore be "my fault". The splat is a 10x10x10 cube, and one might expect that the bounding volume has to be [5.0, 5.0, 5.0, 5.0, 0.0, 0.0, 0.0, 5.0, 0.0, 0.0, 0.0, 5.0] But that seems to be wrong. The one from the screenshot takes the RUB->LUF coordinates of the SPZ-to-glTF into account, and is [-5.0, -5.0, 5.0, 5.0, 0.0, 0.0, 0.0, 5.0, 0.0, 0.0, 0.0, 5.0] but that is also wrong. I've been twiddling with a few axis conversions, and couldn't get it "right" (i.e. matching the splat primitive). But I didn't spend too much time with that, because it's not clear whether the bounding box is wrong, or the rendering/placement of the splat primitive.

What's probably more important:

The orientation of the individual splats seems to be wrong! In the screenshot, the mouse cursor is hovering over the orange splat (orange ... between red and yellow - there's that unit cube again 🙂 ). And this seems to be oriented horizontally. But it should be oriented vertically (because it is that edge of the unit cube). In BabylonJS, this seems to be right.

---

I also noticed that the scaling factor from the tileset model matrix is not applied to the splat when the splat content is in the root tile, but it is applied when the splat content is in a child tile, but that may just be an artifact of the whole ~"matrices" point.

---

Here is an archive with that "Unit Cube Tileset", including the SPZ, GLB, tileset JSON, and Sandcastles for testing (including one with basic tileset modelMatrix editing functionality)

unitCubeTileset-2025-06-21.zip

[keyboardspecialist]

When we tile, we tell SPZ to not do any coordinate conversions, then we do a rotation correction during the glTF load here https://github.com/CesiumGS/cesium/blob/cc8c6e0182f9d5c42cc36d777fa9b523ca56c0a7/packages/engine/Source/Scene/PrimitiveLoadPlan.js#L263

Might try commenting that out and see if your splat orientations render as expected.

[javagl]

I'd need much more time to understand what's going on there. (Sure, it will eventually boil down to something simple, but for now, it is just swizzling some quaternion components, and I'd have to check which axis conversion that is and should be). I've seen some code for coordinate conversions in SPZ. (Yes, I see. I'm a nerd. It can imagine the satisfaction of having figured out this oh-so-clever code the generically do the coordinate conversions. But ... *sigh*). It at least looks like axis conversions might be more complicated than that. I'll probably try it out with this conversion commented out (a bit later today)

[javagl]

I had another short look at this. And I created a test data set to dedicatedly test the rotations:

^{(Here, rendered in BabylonJS)}

It contains, from bottom to top (and left to right)

• red splats, scaled along the x-direction (rotated about [0°, 30°, 60°, 90°] around the y-axis)
• green splats, scaled along the y-direction (rotated about [0°, 30°, 60°, 90°] around the z-axis)
• blue splats, scaled along the z-direction (rotated about [0°, 30°, 60°, 90°] around the x-axis)

Here is a GIF, showing the rendering in CesiumJS, once with applying the rotation in the primitive load plan (mentioned above) and once with not applying it:

Both of them look.... at least different from what BabylonJS is showing (regardless of what's "right" or "wrong" here...).

(My concern is that people might now start generating data that is rendered properly by CesiumJS, but that turns out to be "wrong" in another renderer - I think that this should be carefully reviewed...)

EDIT: Nearly forgot the test data (including tileset and sandcastle):

Cesium Splat Rotation 2025-07-01.zip

[javagl]

I'm not sure if spherical harmonics with degree > 0 are supposed to be supported in the current state. But here's some test data:

Cesium Splats SH 2025-07-02.zip

(With PLY, SPZ, GLB, tileset, and sandcastle)

Rendered in BabylonJS:

And in Cesium:

[javagl]

The issue about the spherical harmonics might be caused by https://github.com/drumath2237/spz-loader/blob/4032056c1d2628ae9cf89dc0c2233cc15381e8ee/packages/core/lib/spz-wasm/gaussianCloud.ts#L48 - to be investigated.

[javagl]

(EDITED:)

From my understanding,

• The spz-loader library calls rotate180DegAboutX on the gaussian cloud
• The GltfSpzLoader does not account for that
• The PrimitiveLoadPlan tries to convert this back by negating the y- and z-coordinates and modifying the rotations

I think that there are several possible inconsistencies here that should be investigated. Is the loader library even supposed to apply this rotation?

Related to the first point of https://github.com/nianticlabs/spz/issues/42

[javagl]

When the same glb content (with SPZ) is referred to by two tiles with different transform, then the positions of the splats are all messed up. Apparently, there is some deduplication happening, so it loads the splats only once, but applies wrong/duplicate transforms to them. This even applies across tilesets.

Test data:

Cesium Splats Tile Translation.zip

[javagl]

It is currently not possible to combine splat- and non-splat data in a single tileset. The check for hasGaussianSplatExtension is wrong. Just because the tileset requires this extension does not mean that each glTF is a splat glTF.

[javagl]

There are many other reasons why it is not possible to mix other content types with splats. Among them is that it is trying to accumulate some data from splat primitives at https://github.com/CesiumGS/cesium/blob/7355e1eba2f0732a694c3e5f03960caa982ff7e1/packages/engine/Source/Scene/GaussianSplatPrimitive.js#L774 based on the selected tiles of the tileset (!), which may or may not contain ... splats, or GLB, or content at all...

However, here's a tileset that combines splats and a GLB

Cesium Splat Rendering Tests 2025-07-17 mix GLB and splat.zip

This is what it should look like:

(Rendering that involved some hacks at the places linked above)

I originally wanted to do some more extensive and systematic coordinate system/rendering tests, but for now, will leave it at that.

[javagl]

It looks like there now is a "version 3" of SPZ, as of https://github.com/nianticlabs/spz/pull/41 . People are using that to generate SPZ files. It is not supported by https://github.com/drumath2237/spz-loader . Not sure how to handle that...

[javagl]

I had a short look at https://github.com/CesiumGS/cesium/issues/12756 , and have to add a few broader, but also more technical and specific thoughts here.

I'm not sure about the role of that tileset.gaussianSplatPrimitive. I think that I see the point, from a very high-level perspective, which is very likely very roughly that ~"all splat tile contents have to be combined into a single 'thing', so that the splats can properly be sorted". But right now, I have to say that I have serious doubts that this can ever "work as intended". Certain quirks, like the one mentioned above, can probably be handled somehow. But there are broader issues with that approach.

On the one hand, one could say that this is a limitation insofar that there ~"can only be one splat data set in a tileset" (even though it may not even be clear what "one splat data set" is). And one could argue that it is necessary to combine all splat data into a single 'thing' anyhow, because even when there are two splat data sets within one tileset, then their splats have to be sorted properly. But this is a weak argument: If this was applicable, then it would raise the question about the case of multiple tilesets with splats, within one scene. And we've already seen some problems when trying to combine multiple splat tilesets in one scene (similar to https://github.com/CesiumGS/cesium/issues/12738 , but also in other cases).

Iff "all the splat data" was supposed to be sorted accordingly, then it would be necessary to move this GaussianSplatPrimitive from the tileset-level to the scene-level. This is very roughly speaking, of course. If the bounding volumes of two such tilesets don't overlap, then one could still sort each of them independently, and just render the (sorted) splat sets, for each tileset, back-to-front. The point is that the responsibility for proper sorting cannot be within the tileset, but has to be at a higher level.

I have not yet read through the details of the GaussianSplatPrimitive. From a quick look, it does seem to have some complexities regarding the state space. For example, when just looking at the variables

  this.gaussianSplatTexture = undefined;
  this._hasGaussianSplatTexture = false;
  this._needsGaussianSplatTexture = true;
  this._gaussianSplatTexturePending = false;

I wonder: Can it ever be the case that hasGaussianSplatTexture===true and needsGaussianSplatTexture===true? Can it ever be the case that gaussianSplatTexture===undefined and hasGaussianSplatTexture===true? The point is: This looks redundant, and redundancy is redundant. Maybe it's possible to make the state space smaller, and be it only via some hasGaussianSplatTexture: { get function() { return defined(this.gaussianSplatTexture); } } or so.

Also... the _ready flag is never set. See https://github.com/CesiumGS/cesium/issues/12756#issuecomment-3102725911

[lilleyse]

• Add support for KHR_gaussian_splatting without SPZ extension
• Remove need to declare gaussian splat glTF extensions in tileset.json

See https://github.com/CesiumGS/cesium/issues/12747#issuecomment-3103348441 for more details.

[GISUser1]

We developed a tool to convert a ply or splat file to KHR_spz_gaussian_splats_compression format 3dtiles. it is in our product GISBox and it is totally free to use. you guys can use it to test this new feature.

[weegeekps]

Add support for KHR_gaussian_splatting without SPZ extension

We are planning to do this and bring everything else up to the latest extension specification revision after SIGGRAPH in time for the September release.

As a part of this, I am planning on putting together a tool to convert old 3D Tiles to the new extension spec.

[ggetz]

We've seem reports of the 3D Tiles Inspector having issues when first connected to a splat tileset using the public API. The workaround for now would be to call the following after the tileset has been loaded.

viewer.cesium3DTilesInspector._viewModel._tileset = tileset;

[javagl]

I don't know where the line https://github.com/CesiumGS/cesium/blob/0a49108a5025feb5e2f6a420ccd58919e39061f3/packages/engine/Source/Scene/ResourceCache.js#L506 is coming from, but when bufferViewId, draco, and spz are defined, then this will not throw.

The test

  it("getVertexBufferLoader throws if bufferViewId and draco and spz are defined", function () {
    expect(() =>
      ResourceCache.getVertexBufferLoader({
        gltf: gltfDraco,
        gltfResource: gltfResource,
        baseResource: gltfResource,
        frameState: mockFrameState,
        bufferViewId: 0,
        primitive: primitive,
        draco: dracoExtension,
        spz: { bufferView: 9999 },
        attributeSemantic: "POSITION",
        accessorId: 0,
        loadBuffer: true,
      }),
    ).toThrowDeveloperError();
  });

should pass, and currently, it doesn't.

(There are zero tests that involve that spz, and the spz parameter is not documented, but this issue is already long enough)