Rendering mesh with multiple uv coordinates per vertex

[D0miH]

Hello, first of all thanks for the awesome library.

I am trying to render a mesh with a texture by first creating a trimesh and then adding TextureVisuals:

tri_mesh = trimesh.Trimesh(vertices=0.001 * mesh.v, faces=mesh.f)
tri_mesh.visual = visual.TextureVisuals(mesh.vt, image=texture)
render_mesh = pyrender.Mesh.from_trimesh(tri_mesh, smooth=True)

However, when I run the code above I get the following error:

ValueError: Incorrect texture coordinate shape

This is because the texture is wrapped around the mesh such that one vertex is mapped to multiple UV coordinates. This is the case because some vertices are mapped to the left/right end of the texture. Therefore the shape of the vertices is (5023,3) and the shape of the vertex coordinates is (5118, 2).

Is there any way I can render this mesh with the texture without an error message?

[patrikhuber]

Hi,

Coincidentally I just got (nearly) exactly the same problem. More generally, we are talking about texture maps with seams here. Importantly, in that case, not only can certain vertices have multiple UV coordinates, the triangle indices for the UV coordinates will also be different from the triangle indices of the "mesh" (vertex values).

It seems to me like pyrender currently does not support this. When creating a Primitive (https://pyrender.readthedocs.io/en/latest/generated/pyrender.Primitive.html#pyrender.Primitive), texcoord_0 has to be equal in length to positions, and it is only possible to specify indices, and not a separate list of texture triangle indices.

In my experience, texture maps with seams are actually quite a common use case (they happen whenever you need to "cut open" (i.e. create a seam) when creating an UV wrapping). @mmatl would you have any advice on how to best accomplish this in pyrender currently, and/or would you consider adding support for texture maps with seams to pyrender?

Thanks a lot.

[jejay]

Hi, I would have opened the same issue if I hadn't found this one. I have n_triangles*3 uv coordinates instead of n_vertices uv coordinates. This representation felt intuitive. I have a lot of duplicates but it would also be way easier to match/align. Can't change it as my textures have seams.

Thanks

[walsvid]

Is there any progress now?

[amundra15]

I am facing the same issue. Do we have a solution now?

[amundra15]

I am facing the same issue. Do we have a solution now?

[amundra15]

I wrote a quick and dirty function to fix my mesh by repeating some of the 3D vertices based on the texture map. Putting it here in case someone is looking for the same.

def fix_mesh_shape(mesh, vt, f, ft):

'''
Add missing vertices to the mesh such that it has the same number of vertices as the texture coordinates
mesh: 3D vertices of the orginal mesh
vt: 2D vertices of the texture map
f: 3D faces of the orginal mesh (0-indexed)
ft: 2D faces of the texture map (0-indexed)
'''

#build a correspondance dictionary from the original mesh indices to the (possibly multiple) texture map indices
f_flat = f.flatten()
ft_flat = ft.flatten()
correspondances = {}

#traverse and find the corresponding indices in f and ft
for i in range(len(f_flat)):
    if f_flat[i] not in correspondances:
        correspondances[f_flat[i]] = [ft_flat[i]]
    else:
        if ft_flat[i] not in correspondances[f_flat[i]]:
            correspondances[f_flat[i]].append(ft_flat[i])

#build a mesh using the texture map vertices
new_mesh = np.zeros((vt.shape[0], 3))
for old_index, new_indices in correspondances.items():
    for new_index in new_indices:
        new_mesh[new_index] = mesh[old_index]

#define new faces using the texture map faces
f_new = ft

return new_mesh, f_new