Copy UV coordinates from a reference mesh.

[ssheorey]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for main branch).

Proposed new feature or change

In many mesh processing workflows, UV coordinates are not preserved (e.g. mesh simplification). UV coordinates are required for many downstream tasks. One way to propagate UV coordinates through mesh processing workflows is by copying them from the original mesh. Here is an outline:

1. Mesh A is processed to mesh B. Mesh A contains UV coordinates (and textures), but mesh B does not have them.
2. Call meshB.InterpolateTextureCoordinatesFrom(meshA). This does:

• Create a ray casting scene with mesh A.
• For each vertex in mesh B, find nearest point in mesh A.
• Assign UV coordinates of nearest point in mesh A to the vertex in mesh B.
• Optionally, copy over any textures from mesh B to mesh A.

References

No response

Additional information

It may be necessary to convert from per-triangle UVs to per-vertex UVs (or vice versa). The following code shows an example of how to make that conversion: https://github.com/isl-org/Open3D/blob/main/cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp#L614

[ssheorey]

Related: #6615

[Amandeep4282]

Greeting, I am new to contributing to Open Source. I would like to give it a try and try to resolve this issue if I may.

Could you please provide me some brief info about the issue and location of files/folder in which it need to be fixed.

[cdbharath]

I implemented this feature and was testing it. I noticed the following anomaly that I am not sure how to resolve.

To test the code, I performed the following

1. Load a sphere mesh
2. Apply FilterSmoothTaubin filter
3. Use InterpolateTextureCoordinatesFrom to interpolate the UV coordinates to the filtered mesh
4. Visualize the original and the filtered mesh

The following images visualizes the textured mesh before and after filtering + UV interpolation Before filtering + UV interpolation:

After filtering + UV interpolation: After filtering and UV interpolation, a single portion of the sphere doesn't have the same texture as the original.

This happens because per-triangle UVs and per-vertex UVs don't map one to one while converting the former to latter. An example is the following texture image from Open3D documentation. The UVs for same mesh vertex could be at the opposite end of the texture image. This results in the entire portion of the image being compressed and assigned to the single stretch of triangles.

How would you suggest I handle this case?

[shanagr]

Disclaimer: I'm new to Open3D as well, so take this as an idea to explore. Someone more experienced can probably help you better.

Assume we have a way to detect if the vertices of a triangle have "crossed the edge and wrapped around". Let, for example, y1 and y2 be two y coordinates in a triangle, and we know y2 has fallen off the edge. Instead of using y1 we can use 1.0 + y1 in our computations (and take the result "modulo" 1.0 in the end once all computations are done). How do we know if something has fallen off the edge? The simplest heuristic I can think of is check if |y1 - y2| > 0.5. If yes, we use 1.0 + min(y1, y2) instead of min(y1, y2).

[dbs4261]

I'm wondering if I am using this correctly. I tried applying this to a mesh that I then applied quadric simplification to and the render didn't come out making sense. The dataset is about 300 MB and is derived from the ODM Test Dataset Sheffield Cross, I did modify it to bake the texutres into a single atlas. I then used this code snippet to try coping the UV coordinates.

int main(int argc, char** argv) {
    const std::string source_path = "combined_mesh.obj";

    open3d::geometry::TriangleMesh mesh;
    if (!open3d::io::ReadTriangleMesh(source_path, mesh, {})) {
        open3d::utility::LogError("Failed to read {}", source_path);
    }

    open3d::geometry::TriangleMesh processed = mesh;
    processed.materials_.clear();
    processed.textures_.clear();
    processed.triangle_material_ids_.clear();
    processed = *processed.SimplifyQuadricDecimation(
            static_cast<std::size_t>(processed.triangles_.size() * 0.5),
            std::numeric_limits<double>::max(), 100.0);

    processed.InterpolateTextureCoordinatesFrom(mesh);
    const std::string out_path = "UVTransfered.obj";
    if (!open3d::io::WriteTriangleMesh(out_path, processed)) {
        open3d::utility::LogError("Failed to write to {}", out_path);
    }
    return EXIT_SUCCESS;
}

Here is a screenshot of the input mesh: And here was the output (after I updated the mtl file):

I was expecting to see some issues around texture seams, but this strange result makes me think I applied it wrong. Thoughts?