.KSplat Color issue with spherical harmonic

[hyqshr]

trafficstars

While rendering my scene with .ksplat with 0 sphericalHarmonicsDegree, it worked well:

        const viewer = new GaussianSplats3D.DropInViewer({
            sharedMemoryForWorkers: false,
            showLoadingUI: true,
            format: GaussianSplats3D.SceneFormat.KSplat,
            sphericalHarmonicsDegree: 0
        });

But when I change the sphericalHarmonicsDegree to 1 or 2, the color seems to be broken:

[mkkellogg]

Out of curiosity, how many splats are in your scene?

[hyqshr]

Only 1 splat.

The broken color issue happens on .ply as well, but only when sh_degree is 1 (0 or 2 is fine). My model is trained from splatfacto-w (https://docs.nerf.studio/nerfology/methods/splatw.html) not sure if this the reason?

[mkkellogg]

Sorry let me rephrase, how many individual splat "blobs" are in your scene? If you press the 'I" key, you'll bring up an info panel that will give you that info:

[hyqshr]

I press "I" and I can not bring up the info panel. Is there other way I could print this?

[hyqshr]

Also - is there a python script to convert ply to ksplat? I think it can help a lot of folk's workflow. I can create a PR if needed

[mkkellogg]

If you want to send me your .ksplat or .ply file, I can tell you how many splats it has. My suspicion is that it has a large number, which can cause issues because there's not enough memory to store spherical harmonics data.

You could also load it in the online viewer here: https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php and if you press 'I' in that viewer, it should bring up the info panel.

[hyqshr]

Below is the info panel. The weird thing is the broken color seems only happens at sh_degree=1 for this KSplat. 0 and 2 are fine

[hyqshr]

I have another model have the same issue when sh_degree is 1 in viewer

[mkkellogg]

That is pretty weird, would you mind sharing your .ply or .ksplat file and I can try to debug?

[hyqshr]

Thank you! Not sure why this link download .ttf format, but I can reproduce the error when I change it to .ksplat

https://forreal.blob.core.windows.net/forreal-model/0/splat-w-office.ksplat?sv=2023-01-03&st=2024-11-22T16%3A33%3A19Z&se=2024-11-23T16%3A33%3A19Z&sr=b&sp=r&sig=YVWvuRTBzwTA4UgOh%2BPyn5UQ9sKhU8EC01dmI4idyr4%3D

[mkkellogg]

Would you also be able to share your original .ply as well? And do you know how many SH degrees should be present in the file?

Also, I get an error when I try to access the above link:

[PaulErpen]

Hi there,

I am experiencing a similar issue. The colors are wrong when limiting the viewer to 1 spherical harmonics band. This happens given a model that has either been trained with the full range of spherical harmonics or a model that has been limited to 1 band. It isn't specific to .ksplat files. I have this issue using .ply files, but it persists when converting to ksplat as well as supersplat-compressed ply files.

The following model has been trained using a maximum sh degree of 1: geo_gaussian-truck-low-1.ply

it looks fine when limiting the sh degree to 0:

but falls apart when the spherical harmonics are set to 1:

Here are some stats about the model:

This also happens for models that are trained using all sh components: mcmc-vsc-truck-low-4_model(2).ply

When limiting to sh 0:

When limiting to sh 1:

But its fine for all sh components (2 obviously):

Here are some stats about the other model:

The supersplat editor seems to display the colors correctly:

I trained these models using different techniques. One is trained using geogaussian's original implementation and the other is trained using gsplat + mcmc. Hope this is helpful :) It would be super appreciated if you find time to further look into this issue. If you need any assistance or further info, please let me know. Thanks for making this awesome project open source.

[mkkellogg]

There is bug that currently manifests when you limit the spherical harmonics to degree 1, and it affects all file types. I have a fix in this branch: https://github.com/mkkellogg/GaussianSplats3D/tree/playcanvas_and_spz_compression

Hopefully I'll get a release out soon :)

[PaulErpen]

Hi thanks for the quick response :) When checking out this branch and running the demo the issue persists:

maybe i am doing something wrong. I did the following:

git clone [email protected]:mkkellogg/GaussianSplats3D.git GaussianSplats3D-fixed
cd GaussianSplats3D-fixed
git fetch
git checkout remotes/origin/playcanvas_and_spz_compression
npm install
npm run build
npm run demo

Then I upload the file mcmc-vsc-truck-low-4_model(2).ply

[hyqshr]

Not sure if this is a related issue - some color problem when SH=2 (The model trained with SH=2):

nerfstudio viewer:

viewer from this repo, SH=2(happens on both raw .ply file, and .ksplat file):

Also viewer from this repo, when I change the SH level to 0:

[mkkellogg]

Sorry for the late response, it turns out I forgot to push some local changes to https://github.com/mkkellogg/GaussianSplats3D/tree/playcanvas_and_spz_compression. Want to give it another try?

[hyqshr]

I just tried again with a new splat, when sh=1 the problem persist. I use the viewer at https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php?art=1&cu=0,1,0&cp=0,1,0&cla=1,0,0&aa=false&2d=false&sh=2, does it include the latest fix?

[mkkellogg]

I totally forgot to update the online viewer, thanks for catching that! It should be fixed now.

[hyqshr]

Just tried again, still broken when sh=1

[mkkellogg]

Can you send my the file you're having issues with?

[hyqshr]

https://forreal.blob.core.windows.net/forreal-model/334/splat.ksplat?sv=2023-01-03&st=2025-02-07T14%3A46%3A30Z&se=2025-02-12T14%3A46%3A00Z&sr=b&sp=r&sig=dPSefgm9z5cHbpfFKulnRhareQJzEuOIQu9Xhf2%2BjFM%3D

[mkkellogg]

Are you doing any rotations or other transformations to the scene? I'm able to view it correctly with SH=1 using the latest in main (which should correspond to v0.4.7)

[hyqshr]

I was just using the web viewer https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php?art=1&cu=0,1,0&cp=0,1,0&cla=1,0,0&aa=false&2d=false&sh=1 with no rotations. Is this viewer most update-to-date?

[mkkellogg]

Interesting, the online viewer is up-to-date. In fact, I can follow the link you posted above, load splat.ksplat, and it all seems to work correctly. What OS and browser are you using?

[hyqshr]

macOS and chrome

On Tue, Feb 11, 2025 at 20:56 Mark Kellogg @.***> wrote:

Interesting, the online viewer is up-to-date. In fact, I can follow the link you posted above, load splat.ksplat, and it all seems to work correctly. What OS and browser are you using?

— Reply to this email directly, view it on GitHub https://github.com/mkkellogg/GaussianSplats3D/issues/375#issuecomment-2652468081, or unsubscribe https://github.com/notifications/unsubscribe-auth/APIDKW3PX2SB5TO4LC3OJJD2PKS6LAVCNFSM6AAAAABSC5T77GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJSGQ3DQMBYGE . You are receiving this because you authored the thread.Message ID: @.***> [image: mkkellogg]mkkellogg left a comment (mkkellogg/GaussianSplats3D#375) https://github.com/mkkellogg/GaussianSplats3D/issues/375#issuecomment-2652468081

Interesting, the online viewer is up-to-date. In fact, I can follow the link you posted above, load splat.ksplat, and it all seems to work correctly. What OS and browser are you using?

— Reply to this email directly, view it on GitHub https://github.com/mkkellogg/GaussianSplats3D/issues/375#issuecomment-2652468081, or unsubscribe https://github.com/notifications/unsubscribe-auth/APIDKW3PX2SB5TO4LC3OJJD2PKS6LAVCNFSM6AAAAABSC5T77GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJSGQ3DQMBYGE . You are receiving this because you authored the thread.Message ID: @.***>