How to use per-face uv-map to texture a mesh?

[trainlock]

Given a trimesh.Trimesh, a per-face uv-map coordinates (not per-vertex), and a texture, I could not find a way to texture the mesh using only trimesh. In trimesh, AFIK, I can only texture a mesh using per-vertex uv-map (not per-face).

I could use the open3d view of the trimesh object to do such an operation. However, the changes are not reflected in the original object, and I am required to write the mesh down to a file and reload it with trimesh in order to get a textured mesh.

import trimesh
import open3d
import numpy as np

from pathlib import Path
from skimage import data
from sklearn.preprocessing import minmax_scale

def _get_arbitrary_uv_map_faces(mesh):
    _arbitrary_uv_map_vertex = minmax_scale(mesh.vertices.view(np.ndarray)[:, :2])
    return np.array([
        _arbitrary_uv_map_vertex[face].reshape(-1)
        for face in mesh.faces.view(np.ndarray)
    ])

def get_textured_mesh(mesh: trimesh, uv_map_faces, texture: np.ndarray):
    # Save texture through open3d and load it again in trimesh
    def _save_load_workaround_to_get_textured_mesh(open3dmesh):
        from tempfile import mkdtemp

        _tmp_fname = Path(mkdtemp()) / "tmp.obj"
        open3d.io.write_triangle_mesh(str(_tmp_fname), open3dmesh)
        return trimesh.load(_tmp_fname)

    # Set uv-coordinates on the mesh and assign texture
    mesh.as_open3d.triangle_uvs = open3d.utility.Vector2dVector(uv_map_faces.reshape(-1, 2))
    mesh.as_open3d.textures = [open3d.geometry.Image(np.asarray(texture)).flip_vertical()]

    # return mesh  # I would expect this to return a textured mesh
    return _save_load_workaround_to_get_textured_mesh(mesh.as_open3d)

def main():
    def transformation(uv_map_faces):  # For illustration purposes, transformation that only works from faces to faces
        return uv_map_faces

    mesh = trimesh.creation.uv_sphere()
    uv_faces = _get_arbitrary_uv_map_faces(mesh)

    scene = get_textured_mesh(
        mesh=mesh,
        uv_map_faces=transformation(uv_faces),
        texture=data.checkerboard(),
    )
    scene.show()

if __name__ == "__main__":
    main()

Is there a way to create a trimesh object out of an open3d object? If not, is there a way to create a trimesh object in memory, without having to write it to a file?

Possibly related issues:

• https://github.com/mikedh/trimesh/issues/1289
• https://github.com/mikedh/trimesh/issues/1117

Expected result:

Outputted result (without workaround):

[mcejp]

I've ran into the same problem and I didn't find a solution other than duplicating the vertices to assign different UV values. You must also pass process=False, maintain_order=True when instantiating the Trimesh.