Function "mesh_normal_consistency" is very slow

[Talmaj]

This is more an enhancement than a bug. Namely the mesh_normal_consistency function is implemented in C++ and is relatively slow when training with a strong GPU. That's why we can see very low GPU utilisation during the execution of mesh_normal_consistency. The effect is so strong that training with an AWS's g4dn.x2large is actually faster than p3.x2large.

[bottler]

In the latest release 0.4.0 of PyTorch3D, we still had a pure python implementation, and this was slower. See for example #252. The C++ code to do the topology calculation (on github only) is a recent enhancement and provides a big speedup, but yes, it does mean that a certain part of the calculation still has to be done on the CPU. Are you definitely not using the released version? It is not obvious how to do the topology calculation in CUDA.

In some applications (e.g. the deform meshes tutorial) the topology doesn't change between calls to mesh_normal_consistency, and actually doesn't need to be recalculated. It would be easy to precalculate it. Would that help your use case?

[Talmaj]

I installed the nightly version, which seems to have the C++ implementation. It is definitely much faster. Before I did not get fully utilised GPU on g4dn.x2large, with the nightly version I get to 100%. However, the nightly version is still too slow for a stronger GPU.

[PabloWiedemann]

Hi there, @bottler I wonder about your comment regarding the pre-computation of the mesh_normal_consistency in the context of the deform meshes tutorial. To my understanding, the mesh_normal_consistency is shape-dependent rather than topology-dependent. It computes the angle between the normals of neighboring faces and those angles could change when deforming a mesh. So, how can one pre-compute this term on a deforming mesh? e.g. in the deform meshes tutorial (To be clear I'm referring to the sphere to dolphin mesh tutorial), the mesh deforms on each iteration. Am I missing something?

Thanks for any help!

[bottler]

mesh_normal_consistency is shape-dependent rather than topology-dependent. It computes the angle between the normals of neighboring faces and those angles could change when deforming a mesh.

It definitely does depend on the angles and therefore on the vertex positions of the mesh. However a large early chunk of the computation time is taken up by working out which faces are adjacent to each face, and this doesn't depend on the vertex positions but only on the faces. So we could imagine a training loop looking like this:

initial_mesh=Meshes(initial_verts, faces)
consistency_layer = MeshNormalConsistency(representative_mesh=initial_mesh)
for iteration in ...:
    ...
    verts = ...
    mesh = Meshes(verts, faces)
    ...
    loss = consistency_layer(mesh) + ...
    ...

where consistency_layer could be faster than the current mesh_normal_consistency.

[PabloWiedemann]

mesh_normal_consistency is shape-dependent rather than topology-dependent. It computes the angle between the normals of neighboring faces and those angles could change when deforming a mesh.

It definitely does depend on the angles and therefore on the vertex positions of the mesh. However a large early chunk of the computation time is taken up by working out which faces are adjacent to each face, and this doesn't depend on the vertex positions but only on the faces. So we could imagine a training loop looking like this:

initial_mesh=Meshes(initial_verts, faces)
consistency_layer = MeshNormalConsistency(representative_mesh=initial_mesh)
for iteration in ...:
    ...
    verts = ...
    mesh = Meshes(verts, faces)
    ...
    loss = consistency_layer(mesh) + ...
    ...

where consistency_layer could be faster than the current mesh_normal_consistency.

I see, that makes total sense :) thank you very much for the explanation!

[patrikhuber]

[...] So we could imagine a training loop looking like this:

initial_mesh=Meshes(initial_verts, faces)
consistency_layer = MeshNormalConsistency(representative_mesh=initial_mesh)
for iteration in ...:
    ...
    verts = ...
    mesh = Meshes(verts, faces)
    ...
    loss = consistency_layer(mesh) + ...
    ...

where consistency_layer could be faster than the current mesh_normal_consistency.

+1 for this, that would definitely be very useful.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[Talmaj]

I am going to prepare a pull request soon.