Get .ply file after training?

[rkakash59]

How can I get .ply file from trained model? I cannot find the .ply in results folder after training. How can I get it from the trained model?

[liruilong940607]

Hi we do not save results in ply format.

[rkakash59]

Can you suggest how to get .ply file from checkpoint file(if possible). I want to use other viewers as well which supports .ply format.

[liruilong940607]

You could take reference from the exporter code in nerfstudio, or the code in the official repo

[Neilstid]

You may add this to Runner class to have .ply output. Then call save_ply when you need ;)

# Experimental
def construct_list_of_attributes(self):
    l = ['x', 'y', 'z', 'nx', 'ny', 'nz']
    # All channels except the 3 DC
    for i in range(self.splats["sh0"].shape[1]*self.splats["sh0"].shape[2]):
        l.append('f_dc_{}'.format(i))
    for i in range(self.splats["shN"].shape[1]*self.splats["shN"].shape[2]):
        l.append('f_rest_{}'.format(i))
    l.append('opacity')
    for i in range(self.splats["scales"].shape[1]):
        l.append('scale_{}'.format(i))
    for i in range(self.splats["quats"].shape[1]):
        l.append('rot_{}'.format(i))
    return l

# Experimental
@torch.no_grad()
def save_ply(self, path):
    os.makedirs(os.path.dirname(path), exist_ok=True)

    xyz = self.splats["means3d"].detach().cpu().numpy()
    normals = np.zeros_like(xyz)
    f_dc = self.splats["sh0"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
    f_rest = self.splats["shN"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
    opacities = self.splats["opacities"].detach().unsqueeze(-1).cpu().numpy()
    scale = self.splats["scales"].detach().cpu().numpy()
    rotation = self.splats["quats"].detach().cpu().numpy()

    dtype_full = [(attribute, 'f4') for attribute in self.construct_list_of_attributes()]

    elements = np.empty(xyz.shape[0], dtype=dtype_full)
    attributes = np.concatenate((xyz, normals, f_dc, f_rest, opacities, scale, rotation), axis=1)
    elements[:] = list(map(tuple, attributes))
    el = PlyElement.describe(elements, 'vertex')
    PlyData([el]).write(path)

[Tiandishihua]

How can I get .ply file from trained model? I cannot find the .ply in results folder after training. How can I get it from the trained model?

Hello! Have you solved the problem of saving as ply files? Thank you .

[Tiandishihua]

You may add this to Runner class to have .ply output. Then call save_ply when you need ;)

# Experimental
def construct_list_of_attributes(self):
    l = ['x', 'y', 'z', 'nx', 'ny', 'nz']
    # All channels except the 3 DC
    for i in range(self.splats["sh0"].shape[1]*self.splats["sh0"].shape[2]):
        l.append('f_dc_{}'.format(i))
    for i in range(self.splats["shN"].shape[1]*self.splats["shN"].shape[2]):
        l.append('f_rest_{}'.format(i))
    l.append('opacity')
    for i in range(self.splats["scales"].shape[1]):
        l.append('scale_{}'.format(i))
    for i in range(self.splats["quats"].shape[1]):
        l.append('rot_{}'.format(i))
    return l

# Experimental
@torch.no_grad()
def save_ply(self, path):
    os.makedirs(os.path.dirname(path), exist_ok=True)

    xyz = self.splats["means3d"].detach().cpu().numpy()
    normals = np.zeros_like(xyz)
    f_dc = self.splats["sh0"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
    f_rest = self.splats["shN"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
    opacities = self.splats["opacities"].detach().unsqueeze(-1).cpu().numpy()
    scale = self.splats["scales"].detach().cpu().numpy()
    rotation = self.splats["quats"].detach().cpu().numpy()

    dtype_full = [(attribute, 'f4') for attribute in self.construct_list_of_attributes()]

    elements = np.empty(xyz.shape[0], dtype=dtype_full)
    attributes = np.concatenate((xyz, normals, f_dc, f_rest, opacities, scale, rotation), axis=1)
    elements[:] = list(map(tuple, attributes))
    el = PlyElement.describe(elements, 'vertex')
    PlyData([el]).write(path)

Hello! I followed your code and found that the saved ply file could not be opened normally, is there any way you can solve it?

[Tiandishihua]

Hi we do not save results in ply format.

大佬，您好！我按照您分享的参考代码来保存成ply结果，发现没办法被原始的gaussian viewer打开查看？您能指点一下吗？非常期待您热心的帮助。

[Neilstid]

Hi, few questions may be to have a better context of that issue:

1. What did you used to open the ply file? Does it work with demo gaussian splating?
2. What is the error message?
3. Where did you put the code? I call save_ply like this: self.save_ply(os.path.join(self.cfg.result_dir, "point_cloud/iteration_{}.ply".format(step))) after the eval step

[Tiandishihua]

Hi, few questions may be to have a better context of that issue:

1. What did you used to open the ply file? Does it work with demo gaussian splating?
2. What is the error message?
3. Where did you put the code? I call save_ply like this: self.save_ply(os.path.join(self.cfg.result_dir, "point_cloud/iteration_{}.ply".format(step))) after the eval step

Hello! It is a great pleasure to receive your enthusiastic response. I used this URL you sent to view the ply file. I saved the ckpt file as a ply file after loading it, and the code looks like this:

splats={} splats["means3d"]=means splats["quats"]=quats splats["scales"]=scales splats["opacities"]=opacities splats["sh0"]=colors[:,:1,:] splats["shN"]=colors[:,1:,:]

Experimental

def construct_list_of_attributes(): l = ['x', 'y', 'z', 'nx', 'ny', 'nz'] # All channels except the 3 DC for i in range(splats["sh0"].shape[1]*splats["sh0"].shape[2]): l.append('f_dc_{}'.format(i)) for i in range(splats["shN"].shape[1]*splats["shN"].shape[2]): l.append('f_rest_{}'.format(i)) l.append('opacity') for i in range(splats["scales"].shape[1]): l.append('scale_{}'.format(i)) for i in range(splats["quats"].shape[1]): l.append('rot_{}'.format(i)) return l

Experimental

def save_ply(save_ply_path):

xyz = splats["means3d"].detach().cpu().numpy()
normals = np.zeros_like(xyz)
f_dc =   splats["sh0"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
f_rest = splats["shN"].detach().transpose(1, 2).flatten(start_dim=1).contiguous().cpu().numpy()
opacities = splats["opacities"].detach().unsqueeze(-1).cpu().numpy()
scale =  splats["scales"].detach().cpu().numpy()
rotation = splats["quats"].detach().cpu().numpy()

dtype_full = [(attribute, 'f4') for attribute in construct_list_of_attributes()]

elements = np.empty(xyz.shape[0], dtype=dtype_full)
attributes = np.concatenate((xyz, normals, f_dc, f_rest, opacities, scale, rotation), axis=1)
elements[:] = list(map(tuple, attributes))
el = PlyElement.describe(elements, 'vertex')
PlyData([el]).write(save_ply_path)

save_ply_path="/app/gsplat/examples/point_cloud.ply" save_ply(save_ply_path)

It's terrible to look at the visualizations. Can you share a copy of your code? My email is [email protected]. Thank you very much for your enthusiastic help and best wishes to you.

[Neilstid]

I didn't spot anything strange in the code you posted.

You can find attached to this the code I used for the ply. Also, once you have your visualization does not hesitate to move around. You may just be in a bad reconstruction spot, and the reconstruction can be good by rotating the splat.

simple_trainer.txt

Hope it will help you :)

[Tiandishihua]

I didn't spot anything strange in the code you posted.

You can find attached to this the code I used for the ply. Also, once you have your visualization does not hesitate to move around. You may just be in a bad reconstruction spot, and the reconstruction can be good by rotating the splat.

simple_trainer.txt

Hope it will help you :)

Thank you very much for your kind answer, I have solved this problem, best wishes to you.

[NIZAHO]

我没有在您发布的代码中发现任何奇怪的东西。 你可以找到我用于铺层的代码。此外，一旦你有了你的可视化，就毫不犹豫地四处移动。您可能只是处于一个糟糕的重建点，而通过旋转 splats 可以很好地重建。 simple_trainer.txt 希望它能帮助你:)

非常感谢您的友好回答，我已经解决了这个问题，向您致以最良好的祝愿。

请问您是怎么解决的呢？我使用这个代码得到的.ply会产生大量的飞影，它几乎重建了它看到的一切

[Master-cai]

Hi guys! I have used the code to convert the splat to SIBR style, but the viewer doesn't work properly. The SIBR viewer discards some gaussians near the cameras, but works normal in other two model: "Initial Points" and "Ellipsoids". I don't know what's going on. The following is some examples when snapping to camera 0:

in Splats model

in Initial Points model

in Ellipsoids model

[Sagnik2810Sarkar]

I didn't spot anything strange in the code you posted.

You can find attached to this the code I used for the ply. Also, once you have your visualization does not hesitate to move around. You may just be in a bad reconstruction spot, and the reconstruction can be good by rotating the splat.

simple_trainer.txt

Hope it will help you :)

Hey I was attempting to use the version of the simple_trainer file you provided but there seems to be a function you're importing from examples/utils.py that is missing from mine - normalized_quat_to_rotmat

Mind helping me out in defining this or forwarding your utils.py file so i can see what this function needs to do? Appreciate any help : )

[Neilstid]

Hi, the normalized_quat_to_rotmat is now in utils.py from gsplat (https://github.com/nerfstudio-project/gsplat/blob/da0a201b8eafacb127fd8f09c56f2989b453a9ab/gsplat/utils.py#L8). Changing the reference should do it :)

[HirokiMorita]

When I ran the code, I got the following error

Traceback (most recent call last):
  File "C:\Users\10028990\Documents\GenAI\3D\gsplat2\gsplat\export.py", line 990, in <module>
    main(cfg)
  File "C:\Users\10028990\Documents\GenAI\3D\gsplat2\gsplat\export.py", line 966, in main
    runner.splats[k].data = ckpt["splats"][k]
KeyError: 'means3d'

I checked the KEY of the loaded CKPT file and it does not contain means3d. dict_keys(['step', 'splats'])

Is there a problem with the way you are making them learn?

[HirokiMorita]

Should I change the following part of simple_trainer to also save self.splats[“means”]?

            if step in [i - 1 for i in cfg.save_steps] or step == max_steps - 1:
                mem = torch.cuda.max_memory_allocated() / 1024**3
                stats = {
                    "mem": mem,
                    "ellipse_time": time.time() - global_tic,
                    "num_GS": len(self.splats["means"]),
                }
                print("Step: ", step, stats)
                with open(
                    f"{self.stats_dir}/train_step{step:04d}_rank{self.world_rank}.json",
                    "w",
                ) as f:
                    json.dump(stats, f)
                data = {"step": step, "splats": self.splats.state_dict()}
                if cfg.pose_opt:
                    if world_size > 1:
                        data["pose_adjust"] = self.pose_adjust.module.state_dict()
                    else:
                        data["pose_adjust"] = self.pose_adjust.state_dict()
                if cfg.app_opt:
                    if world_size > 1:
                        data["app_module"] = self.app_module.module.state_dict()
                    else:
                        data["app_module"] = self.app_module.state_dict()
                torch.save(
                    data, f"{self.ckpt_dir}/ckpt_{step}_rank{self.world_rank}.pt"
                )

[Neilstid]

Well, correct me if I'm wrong, but this is not related to ply file and should therefore be placed in either the correct issue or open a new one.

Anyway, there's should not be any error. self.splats[“means”] should be saved by self.splats.state_dict(). So maybe there's no splat? But this is hard to tell. You should definitly open an issue and check that in checkpoint file there is at least one splat (this should contain means). Also check that the checkpoint and the code you are running are on the same version of the code.

[HirokiMorita]

It seems that the name of the Key was changed in the latest code, and it worked once I updated that! Thank you very much for your detailed response!

[Nicous20]

very nice!! but now shoud change the means3d to means!

[ZhouCX117]

@Master-cai Could you please share the code converting the splat to SIBR style?

[felixniemeyer]

If anyone still looking for a way to output as .ply: It seems utils now implements a save_ply function and simple_trainer.py supports it via the -save_ply argument.

[maturk]

If anyone still looking for a way to output as .ply: It seems utils now implements a save_ply function and simple_trainer.py supports it via the -save_ply argument.

Yes, if using simple_trainer.py you can use the flag --save-ply to save to disk. Note, ply file sizes can be large, so I set this to false at default. Example command: python simple_trainer.py mcmc --data-dir ./360_v2/garden/ --save-ply --ply-steps 20000

[liruilong940607]

already supported in latest commit so closing it.