SVG <-> vertex mapping is off when vertices are left out of flatmap

[Gilles86]

@tknapen and me are working on retinotopic maps at very high resolution (0.7 - 1mm isotropic and below). To visualize the results, we would like to flatten just the most posterior part of the occipital pole as a circle. Specifically, I used Freesurfer's tksurfer to cut of a large part of the cortex, after which I flatten it with mris_flatten. The results looks like this:

However, I noticed that when I draw ROIs on the resulting overlays.svg, they end up somewhere else on the surface rendered by webgl.

By some experimenting, I noticed that the amount of shifting is related to the following property: the resulting flattened cortical maps were smaller than the SVG. In other words: when there is a white margin between the borders of the flatmap and the image canvas.

I could make the mask fall on the right place in the webgl rendering by resizing the pngs to exactly match the outline of the image canvas. However, when I then go to voxel space, the masks are still (completely) off.

I then delved a bit deeper into pycortex's code and noticed the following things: a) The mapping between vertices and the SVG seems to rely on a coordinate system where the minimum and maximum coordinates of the SVG canvas correspond to the minimum and maximum of the vertices in 2D. b) This range is defined in svgoverlay. get_overlay and the initialization of svgoverlay. SVGOverlay. Importantly, it depends on all points that are defined in the flat surface, _including those points that were not included in the flattening procedure and are not in in the 2D plane. However, their x-y coordinates can fall out off the range of the points https://github.com/gallantlab/pycortex/blob/f1cd6abb5e193d0b7ec120107268dbbbf3a002fc/cortex/svgoverlay.py#L629

When not all points are used in the flattening procedure, like in my case, they can linger on in the code that defines the range of the SVG and thereby mapping to the surfaces/voxels.

It seems that someone else had similar problems before since this code with a valid-mask lingers here and there, but is not really used https://github.com/gallantlab/pycortex/blob/f1cd6abb5e193d0b7ec120107268dbbbf3a002fc/cortex/svgoverlay.py#L620

I wrote a small patch that seem to fix the problem for us here. However, I know too little about pycortex's infrastructure to know if this could break something: https://github.com/Gilles86/pycortex/commit/551e59b7b2715d06746e6fed99694b6d5e23d7fc

So to the maintainers: a) Is the use flat maps of a limited part of the cortex something you want as a feature of pycortex? b) How can we safely implement this in the master branch?

[marklescroart]

Thanks for raising this issue! We are having some internal conversation about it; even the creators / maintainers can't necessarily see immediately what will cause problems (I think this is the case for any sufficiently complex piece of code...). We will get back to you soon(ish)...

[Gilles86]

Cool, thank you!

Maybe important to mention that my patch https://github.com/Gilles86/pycortex/commit/551e59b7b2715d06746e6fed99694b6d5e23d7fc still does not work properly: the masks now get properly rendered in webgl, but the SVG <-> vertex mapping that you can get from get_roi_verts is off: it seems as if it is confusing the indices of flat and flat[valid]. Consequently, also the conversion to voxel space is off.