Artifacts when visualizing Atlas Structure Meshes

[nickdelgrosso]

Hi,

I tried doing some visualization of the atlas meshes in Napari, which takes in the mesh vertices (mesh.points) and the mesh faces (mesh.cells[0].data), and noticed quite a bit of artifacting when the mesh is rotated in the viewer.

This goes away when the mesh is in one direction:

But all the faces disappear when the mesh is rotated 180 degress from that:

After digging into it a bit (I thought at first that it was a napari rendering issue), I think maybe it has to do with the vertex index order in the face data. This seems to be a problem even when the original .obj file is loaded with the original normals data (normal vectors visualized below, they look quite good):

But when the lighting is tested in blender, a similar problem occurs (looking differently because of the different lighting model, but the same orientation behavior relative to the point light happens):

Recalculating the normals in blender produces a much more even lighting, but new artifacts appear as well:

In short, I think that if the mesh data were either saved or loaded differently, artifacts in the 3D mesh visualization would disappear.

I hope this was clear. To recreate it and explore the artifacts, here's a script that recreates it:

import numpy as np
import napari
from bg_atlasapi import BrainGlobeAtlas

with napari.gui_qt():

    viewer = napari.Viewer(ndisplay=3)

    atlas = BrainGlobeAtlas("allen_mouse_25um")
    mesh = atlas.mesh_from_structure("CH")

    verts = (mesh.points - mesh.points.mean(axis=0)) / 100
    faces = mesh.cells[0].data
    values = np.ones(len(mesh.points))

    viewer.add_surface(name="CH", data=(verts, faces, values), opacity=0.3)

[FedeClaudi]

interesting.. have you tried with the mesh you get directly from the allen atlas (as in not through the API)? I remember looking at a couple of them in blender and they looked fine, if that's the case then we know that we've introduced these artifacts during the atlas generation process.

I think maybe it has to do with the vertex index order in the face data.

What do you mean? If the vertexes and faces look fine and the normals are correct I'm not sure what's going on..

[nickdelgrosso]

In blender, I loaded the obj file from ~/.brainglobe/allen_mouse_25um_v1.2/meshes. I'm not sure if this is directly from the allen atlas; if you could point me to a better file to try, though, that would be great.

What do you mean? If the vertexes and faces look fine and the normals are correct I'm not sure what's going on..

The normal vectors aren't sent to napari, just the vertex and face indices. In many basic 3D rendering programs, the clockwise or anti-clockwise order of the face indices is used to infer the normal direction; I'm guessing from these artifacts that the calculation of the face order in creating these meshes used set them relative to a view axis, rather than relative to the faces themselves. But I'm not sure why blender, which has access to the normals data in the .obj file, isn't overriding it in the lighting settings; my guess was that there is an interaction between the face indices and the normal vector, but now I'm not so sure. In any case, something is off about these files.

[nickdelgrosso]

Exploring this further, it looks like the split normals are related to these artifacts. Visualizing them as violet lines (vertex normals in cyan):

The split normals are all pointing in the same direction. This can't be an artifact; they are all too precise. It must be calculated.

To test this, I removed the normal data from the .obj file (all the lines beginning with vn) and re-loaded the mesh into blender. No artifacts!

This still does not answer why the napari example is off, but it's a first step. Ideas?

[FedeClaudi]

What are split normals?

Anyway, one thing you can try is, if you load the .obj in python using vedo the resulting Mesh object should have a method to recalculate normals. You can try doing that and saving back to file to see if that fixes it. Another thing that might be worth trying is to save the Mesh object in a different file format (e.g. .stl), it might be that there's a bug specific to writing .objs

[nickdelgrosso]

Split normals are custom normals supplied by the developer; they aren't calculated from the software. Described here: https://docs.blender.org/manual/en/latest/modeling/meshes/structure.html#custom-split-normals. Turning off the split normals view as described in that link also removes the artifacts.

I don't think that recalculating the normals would be a good idea; depending on what approach is used, it might solve some problems, but can also introduce new artifacts. Since the surface data is already quite good, it would be great to just use what's there.

Agreed that it's probably a bug in how the OBJ files are created (I noticed, for example, that the face data is using single-integer values rather than the three-index syntax for associating vertex, normal, and texture data, in this case, would look like this: f 1//1 2//2 3//3), but I don't know if that's an actual bug.

I haven't seen the original data to verify where the problem is (whether in the original data, the transformation steps, or the writing of obj files); where would be a good place to look?

[vigji]

Hey Nick, happy that you are looking a bit into these files! I am not an expert on working with meshes, so can't provide many pointers about what exactly is producing these results. But if you want to have a look at where exactly the meshes are parsed from the allen servers and saved into the atlas data we deploy for bg-atlasapi usage, it is here. We use the meshio library for reading and writing the obj files after some rescaling and reorientation if needed (none should happen for the Allen mouse meshes though)