trimesh
trimesh copied to clipboard
Is it possible to know which UV pixel point would be a 3d vertex position on a 3d model?
Hello, I am trying to figure out which UV pixel point(2d) would be a 3d position on a 3d model. Is it possible? (The 3d position to a pixel point is also okay.) I know three pixel points for the vertex points on the triangular mesh, but I want to know the UV pixel values for points inside the triangle.
Yes, it is possible.
UV -> 3D
write a position map :
def compute_interpolation_map(shape, tcoords, values):
points = (tcoords * np.asarray(shape)[None, :]).astype(np.int32)
x = np.arange(shape[0])
y = np.flip(np.arange(shape[1]))
X, Y = np.meshgrid(x, y)
res = griddata(points, values, (X, Y), method='cubic')
res[np.isnan(res)] = 0
return res
mesh = trimesh.load(path, process=False)
image_size = (512, 512)
position_map = compute_interpolation_map(image_size, mesh.visual.uv, mesh.vertices)
Note that compute_interpolation_map
lets you create a map by linearly interpolate any per-vertex value (e.g. normal vector)
3D -> UV
mesh = trimesh.load(path, process=False)
# Get the corresponding mesh triangles
tids = trimesh.proximity.closest_point(mesh, points)[2]
# Get the barycentric coordinates of the points in their respective triangle
def barycentric_coordinates_3d(vertices, points):
v0 = vertices[:, 1, :] - vertices[:, 0, :]
v1 = vertices[:, 2, :] - vertices[:, 0, :]
v2 = points - vertices[:, 0, :]
d00 = np.einsum('ij,ij->i', v0, v0)
d01 = np.einsum('ij,ij->i', v0, v1)
d11 = np.einsum('ij,ij->i', v1, v1)
d20 = np.einsum('ij,ij->i', v2, v0)
d21 = np.einsum('ij,ij->i', v2, v1)
denom = d00 * d11 - d01 * d01
barys = np.zeros_like(points)
barys[:, 1] = (d11 * d20 - d01 * d21) / denom
barys[:, 2] = (d00 * d21 - d01 * d20) / denom
barys[:, 0] = 1.0 - barys[:, 1] - barys[:, 2]
return barys
#barys = barycentric_coordinates_3d(mesh.vertices[mesh.faces[tids]], points)
barys = trimesh.triangles.points_to_barycentric(mesh.vertices[mesh.faces[tids]], points, method='cramer') # almost 3 times faster !
# Finally, the uv coordinates
uvs = np.einsum('ij,ijk->ik', barys, mesh.visual.uv[mesh.faces[tids]])
I did not test anything but it should work
Thanks for the writeup @Kiord!
Just wondering, does the above barycentric_coordinates_3d
perform better than trimesh.triangles.points_to_barycentric
?
https://github.com/mikedh/trimesh/blob/52fded0ada210358dacef8a8b1de82d71615ade6/trimesh/triangles.py#L468-L497
Hello @mikedh, thanks for all the great work !
I should have searched a bit more in Trimesh because I did not know this function was already implemented. I made a simple benchmark :
trimesh.triangles.points_to_barycentric(method='cramer'): 2.128999 ms for 100 tests.
trimesh.triangles.points_to_barycentric(method='cross') : 4.126702999999999 ms for 100 tests.
barycentric_coordinates_3d : 5.631712999999996 ms for 100 tests.
barycentric_coordinates_3d
is the slowest of the 3 methods so @daqjjang should use trimesh.triangles.points_to_barycentric
with method='cramer'
.
Thank you for pointing this out !
To go back to the original question
The first UV->3D solution that I proposed is not optimal if you have only a few queries to do or if you need to get precise 3D positions (they will be limited by the image resolution).
Instead, another solution is to use the barycentric coordinates (as mentioned in the 3D->UV solution):
tids = trimesh.proximity.closest_point_uv(mesh, uv)
barys = trimesh.triangles.uv_to_barycentric(mesh.visual.uv[mesh.faces[tids]], uv)
points = np.einsum('ij,ijk->ik', barys, mesh.vertices[mesh.faces[tids]])
AFAIK, these functions are not implemented in Trimesh and a PR could be made. I doubt that it is very useful though.
EDIT We would need :
- An attribute
triangles_uv_tree
similar totriangles_tree
but with 2d coordinates intrimesh.Trimesh
class. - A function
trimesh.proximity.nearby_faces_uv
similar totrimesh.proximity.nearby_faces
- A function
trimesh.proximity.closest_point_uv
similar totrimesh.proximity.closest_point
- A function
trimesh.triangles.uv_to_barycentric
similar totrimesh.triangles.points_to_barycentric
- A lot of code can be factorized between the "uv" version and "3d" version of these.
Thank you so much!
Yes, it is possible.
UV -> 3D
write a position map :
def compute_interpolation_map(shape, tcoords, values): points = (tcoords * np.asarray(shape)[None, :]).astype(np.int32) x = np.arange(shape[0]) y = np.flip(np.arange(shape[1])) X, Y = np.meshgrid(x, y) res = griddata(points, values, (X, Y), method='cubic') res[np.isnan(res)] = 0 return res mesh = trimesh.load(path, process=False) image_size = (512, 512) position_map = compute_interpolation_map(image_size, mesh.visual.uv, mesh.vertices)
Note that
compute_interpolation_map
lets you create a map by linearly interpolate any per-vertex value (e.g. normal vector)3D -> UV
mesh = trimesh.load(path, process=False) # Get the corresponding mesh triangles tids = trimesh.proximity.closest_point(mesh, points)[2] # Get the barycentric coordinates of the points in their respective triangle def barycentric_coordinates_3d(vertices, points): v0 = vertices[:, 1, :] - vertices[:, 0, :] v1 = vertices[:, 2, :] - vertices[:, 0, :] v2 = points - vertices[:, 0, :] d00 = np.einsum('ij,ij->i', v0, v0) d01 = np.einsum('ij,ij->i', v0, v1) d11 = np.einsum('ij,ij->i', v1, v1) d20 = np.einsum('ij,ij->i', v2, v0) d21 = np.einsum('ij,ij->i', v2, v1) denom = d00 * d11 - d01 * d01 barys = np.zeros_like(points) barys[:, 1] = (d11 * d20 - d01 * d21) / denom barys[:, 2] = (d00 * d21 - d01 * d20) / denom barys[:, 0] = 1.0 - barys[:, 1] - barys[:, 2] return barys #barys = barycentric_coordinates_3d(mesh.vertices[mesh.faces[tids]], points) barys = trimesh.triangles.points_to_barycentric(mesh.vertices[mesh.faces[tids]], points, method='cramer') # almost 3 times faster ! # Finally, the uv coordinates uvs = np.einsum('ij,ijk->ik', barys, mesh.visual.uv[mesh.faces[tids]])
I did not test anything but it should work
How to save uv coordinates (uvs) as an image? Thanks.
An errorr pops up from using "mesh.visual.uv" the error is: AttributeError: 'ColorVisuals' object has no attribute 'uv'
How can I get those points from this line?
tids = trimesh.proximity.closest_point(mesh, points)[2]
I assume those are not the vertices from
mesh = trimesh.Trimesh(
vertices=vertices_numpy,
faces=faces_numpy,
vertex_colors=color_numpy,
)