deformation fields as manipulatable numpy arrays?

[samtorrisi]

Hi ANTsPy experts-

I'm pretty new to ANTsPy and got some basic SyN registrations to work well. I'm now trying to synthesize geometric distortions by manually manipulating a diffeomorphic deformation field in numpy and apply it back to the original anat with ants.apply_transforms() to distort it. I got the general plan to work with AFNI's 3dQwarp (which also outputs 3-volume, xyz deformation fields) but I want to use ANTsPy for this project. Unfortunately my struggle is basically I/O and I'm hoping you can shed some light.

in descriptive form, I'm:

loading two 3D anatomies with ants.image_read() resampling to match with ants.resample_image_to_target() running ants.registration() with an outprefix to save the .mat and deformation field (.nii.gz) importing the deformationfield.nii.gz back into python with nibabel.load() and nibabel.get_fdata()

this has deformation field has 5 dimensions: importedback.shape (120, 120, 100, 1, 3)

so i squeeze this matrix into 4 dims, split that into 3 vols, manually manipulate with numpy-friendly tools, bring the vols back together and try to apply it to the original anatomy. i was using ants.apply_transforms(), however i'm not quite sure how to populate the transformlist option (if this is now even possible) and my attempts are breaking.

this procedure seems really kludgey and so, despite it breaking at the end, am i big-picture missing a way to do this only with ANTsImage objects and not having to export and re-import with nibabel? let me know if you want more info or error outputs, or if you can provide any clues or thoughts at all which would be so appreciated. thank you!

-Sam

[ntustison]

To point you in the right direction, what are you doing in terms of synthesizing the warp fields? I'm wondering if there are tools in ANTsPy which might accomplish what you are trying to do outside ANTsPy.

[samtorrisi]

thanks for the rapid reply! i've used antspynet.utilities.data_augmentation() but the warps aren't local or extreme enough. i'd like full control of that aspect, in the form of simple x,y,z numpy arrays. if possible. can ANTsPy nonlinear transforms be extracted as such?

[gdevenyi]

You may want to investigate SimpleITK, which offers comprehensive interfaces for loading/saving data, and careful conversion to/from NumPy https://simpleitk.readthedocs.io/en/master/index.html

[stnava]

you can generate pretty extreme deformations in antspy. here is an example:

import ants
import antspynet
import ants
image1_list = list()
image1_list.append(ants.image_read(ants.get_ants_data("r16")))
image2_list = list()
image2_list.append(ants.image_read(ants.get_ants_data("r64")))
input_segmentations = list()
input_segmentations.append(ants.threshold_image(image1_list[0], "Otsu", 3))
input_segmentations.append(ants.threshold_image(image2_list[0], "Otsu", 3))
input_images = list()
input_images.append(image1_list)
input_images.append(image2_list)
data = antspynet.randomly_transform_image_data(image1_list[0],
    input_images, input_segmentations,
    sd_affine=1,
    sd_smoothing=7.0,
    sd_noise=10.,
    number_of_random_points = 200,
    deformation_transform_type = 'exponential',
    transform_type = "deformation" )
ants.plot( data['simulated_images' ][0][0] )

you would change parameters to get what you want.

if you just want a numpy array, you can do ( there are other ways as well ):

fi = ants.image_read(ants.get_ants_data('r16'))
mi = ants.image_read(ants.get_ants_data('r64'))
mytx = ants.registration(fixed=fi, moving=mi, type_of_transform = 'SyN' )
mydef = ants.image_read( mytx['fwdtransforms'][0] )
mydef.numpy()

[samtorrisi]

this is really helpful, thank you @stnava your first example was fun but still not quite what i want. i beefed up your second and got the 2D version to work:

import ants import nibabel as nib import numpy as np fi = ants.image_read(ants.get_ants_data('r16')) mi = ants.image_read(ants.get_ants_data('r64')) mytx = ants.registration(fixed=fi, moving=mi, type_of_transform = 'SyN' ) mydef = ants.image_read(mytx['fwdtransforms'][0]) mydefnump = mydef.numpy() #slab of artificial warp blank = np.zeros(mydefnump.shape) blank[120:140,:,0] = 5 together = mydefnump + blank makenii = nib.Nifti1Image(together, np.eye(4)) nib.save(makenii, 'together.nii.gz') mytx['fwdtransforms'][0] = 'together.nii.gz' mywarpedimage = ants.apply_transforms( fixed=fi, moving=mi, transformlist=mytx['fwdtransforms'] ) ants.plot(mywarpedimage)

However i can't for the life of me get this to work with 3D data:

#moving image anatomy mi = nib.load('subjectT1.nii.gz') mifromnibabel = ants.from_nibabel(mi) #fixed image template fi = nib.load('MNI152_2009_template.nii.gz') fifromnibabel = ants.from_nibabel(fi) affine = fi.affine resampMi = ants.resample_image_to_target(mifromnibabel, fifromnibabel) mytx = ants.registration(fixed=fifromnibabel, moving=resampMi, type_of_transform = 'SyN' ) mydef = ants.image_read(mytx['fwdtransforms'][0]) mydefnump = mydef.numpy() #3D slab of artificial warp blank = np.zeros(mydefnump.shape) blank[110:140,:,105:125,0] = 5 together = mydefnump + blank makenii = nib.Nifti1Image(together, affine) nib.save(makenii, 'fieldTogether.nii.gz') mytx['fwdtransforms'][0] = 'fieldTogether.nii.gz' #i've also given full path here mywarpedimage = ants.apply_transforms( moving=resampMi, fixed=fifromnibabel, transformlist=mytx['fwdtransforms']) ants.plot(mywarpedimage, axis=2, nslices=24)

"fieldTogether.nii.gz" outputs fine and i can see my slab, but i'm struggling to apply the transform. ants.plot shows the subject anatomy but it hasn't been warped. beating my head against the wall, i've tried so many things but have probably overlooked something dumb (sorry in advance).

-sam

[stnava]

I am not sure what you are trying to do here but the first thing that might help would be to get rid of the nibabel calls. just use ants.image_read ants.image_write and see if that clarifies anything.

[samtorrisi]

ok removing the nibabel calls cleaned things up, thanks. but same behavior. then i started varying the values of the artificial slab i was injecting. apparently the 2D version of my code does what i expected and creates a nice sharp line of deformation, however something different appears to be going on in 3D and it was quite subtle and so i didn't notice that apply_transforms() was actually working. however, even when now making the slab values extreme, the outputs change but are still not at all what i expect (e.g. i'm seeing no sharp lines now and nonlinear effects seem to propagate across the brain). so actually i've gone from thinking this was an I/0 issue to simply i-don't-understand-ANTs-transforms-enough-yet. i have some reading to do. in the meantime thanks for helping me along!