transforming an image does not update it's header

[rueberger]

image.direction is not updated after I apply an affine transform to image. This could be argued to be reasonable behavior, were the affine transform tracked behind the scenes. But evidently this is not the case, as ants.transform_index_to_physical_point does not change upon application of the affine transform. Which, unless I am missing something, is a significant bug.

In other words, I need the transform between index and physical space resulting from the application of a series of affine and displacement field transforms to some image.

> ants.transform_index_to_physical_point(image, [1, 2, 3])
[.1, .2, .3]
> ants.transform_index_to_physical_point(nonzero_affine.apply_to_image(image), [1,2,3])
[.1, .2, .3]

I must be missing something, I expect this is a fairly common use case

[stnava]

brief summary:

• the header describes the physical space associated with a set of pixels/voxels
• the affine transformation is a mapping either within or between domains - but it does not know anything about images
• when resampling an image, one must define the output domain which tells how the points go back to voxel-space
• consequently, the affine transformation doesn't impact the physical space of the image.

[stnava]

we use composite transforms to encapsulate multiple transformations applied in series. it's important to keep track of them carefully and check that compositions are correct if you are using them extensively.

[rueberger]

Hmm, I'm confused. My intuition is that an affine transform should affect the mapping to voxels-space and that a displacement field certainly should.

Let's say that I have a pipeline that computes a SyN registration in ANTs followed by a third-party registration that outputs a displacement field, and I wish to compute the equivalent coordinate transform. To do that I need a transformation from the original moving image's physical space to the index space of the transformed moving image. How might I obtain that transformation?

[stnava]

ok - maybe you might gain insight from this: https://simpleitk.readthedocs.io/en/master/fundamentalConcepts.html

I don't know what you would do outside of ITK or ANTs.

[rueberger]

Thanks. I've read the docs and perused some of the notebooks but it's not yet clear to me how to do this. Could you comment as to whether it's possible to do this in ANTs? Is the question clear?

cross-modality registration. https://arxiv.org/abs/2004.10282

[ntustison]

I'm guessing that you want the affine transform to be propagated to the header such that the image in your initial example is actually transformed by the specified affine transform. But that is not what this call nonzero_affine.apply_to_image(image) is doing. It's actually using image as the reference domain (in terms of physical space) and transforming the contents (i.e., voxel intensities) of image to the reference domain (itself) using nonzero_affine. So your index in the reference domain is the same before and after the call nonzero_affine.apply_to_image call although the voxel intensity at that index will potentially be different.