SegAnyGAussians icon indicating copy to clipboard operation
SegAnyGAussians copied to clipboard

How should we use this result to get a mesh from the sugar model?

Open hanjoonwon opened this issue 1 year ago • 6 comments

https://github.com/Anttwo/SuGaR

When i get mesh from sugar model, i run with this task to get only the object i want and get pt file. how should i apply it to sugar model?

hanjoonwon avatar Mar 15 '24 10:03 hanjoonwon

Hi, can you provide more information about what you did? The .pt file provided by SAGA is a binary mask for 3D Gaussians, i.e., 0 means background and 1 means foreground.

Jumpat avatar Mar 18 '24 02:03 Jumpat

u did? The .pt file provided by SAGA is a binary mask for 3D Gaussians, i.e., 0 means background and 1 means foreground.

@Jumpat
You recommended the current repository when I posted an issue about how to remove everything but the objects from the sugar model and just get the object mesh.

So I'm wondering if I can apply this segmentation to the sugar model and get just the object mesh I want?

hanjoonwon avatar Mar 18 '24 04:03 hanjoonwon

I think yes, since you can use SAGA to obtain the vertexes of the target object. Then you just need to retrieve all edges belong to it. However, you may need to implement it by yourself. I don't know whether sugar keeps the original attributes of Gaussians. I think this wouldn't be too hard.

Jumpat avatar Mar 18 '24 06:03 Jumpat

I think yes, since you can use SAGA to obtain the vertexes of the target object. Then you just need to retrieve all edges belong to it. However, you may need to implement it by yourself. I don't know whether sugar keeps the original attributes of Gaussians. I think this wouldn't be too hard.

Thank you for your response. For now, I can only get the segmented 3dgs rendering result, not the mesh.

hanjoonwon avatar Mar 18 '24 11:03 hanjoonwon

@Jumpat @hanjoonwon did you have the chance to think about this further? I ended up here from google looking for a way to build upon sugar + saga. Any hints on how to get a step closer to this, e.g. getting the meshes from a mask are welcome. I'm happy to supply you with a model if that's what's needed to think about this further.

obtain the vertexes of the target object.

could you please point out how to get the vertex (coordinates?) and how to check if the two 3DGS models (SAGA+SUGAR) are aligned and in the same scale. I'm happy to code this, I'll just need an initial direction.

chris-aeviator avatar Mar 12 '25 18:03 chris-aeviator

@Jumpat @hanjoonwon did you have the chance to think about this further? I ended up here from google looking for a way to build upon sugar + saga. Any hints on how to get a step closer to this, e.g. getting the meshes from a mask are welcome. I'm happy to supply you with a model if that's what's needed to think about this further.

obtain the vertexes of the target object.

could you please point out how to get the vertex (coordinates?) and how to check if the two 3DGS models (SAGA+SUGAR) are aligned and in the same scale. I'm happy to code this, I'll just need an initial direction.

Hello, as of now, we have not planned to integrate SAGA with surface reconstruction methods such as Sugar and 2D-GS, as this functionality is not a primary feature of SAGA. However, implementing this integration should be feasible. To obtain the coordinates, you would need to utilize the segmentation results—the bitmap that identifies whether a 3D Gaussian is part of the target object—to extract the corresponding 3D Gaussian xyz coordinates from the 3D-GS model. Check here.

Jumpat avatar Mar 13 '25 06:03 Jumpat