ANTsR
ANTsR copied to clipboard
ANTsX
antsRegistration does not produce equivalent results
Multiple runs of antsRegistration() on the same images produces different results. Here is a simple example:
library(ANTsR)
template <- antsImageRead(getANTsRData("ch2")) #colins template
atlas <- antsImageRead(getANTsRData("mnia")) #aal atlas
#register template space to atlas
regRigid <- list()
regRigidcount <- NA
for(i in 1:10){
regRigid[[i]] <- antsRegistration(fixed = atlas, moving = template, typeofTransform = c("Rigid"))
regRigidcount[[i]] <- sum(regRigid[[i]]$warpedmovout)
print(regRigidcount[[i]])
}
# the counts are not the same for the 10 runs of the registration
Similarly non-identical results show up for other transforms ("Affine", "SyNCC"). Although the differences seem somewhat small, it is not clear why they are different at all. More importantly, these small differences have substantive consequences down-stream: we ran a lesion-symptom mapping analysis and wanted to describe the results in terms of the AAL atlas, but the 10 somewhat different transformations produce somewhat different descriptions:
#read in data to cluster
# file is available from http://drive.google.com/file/d/1tm2PI6yxhEngB60jNjoP6YQ3T_H7u8JM"
data <- antsImageRead("LSM_result.nii")
#Rigid Transformation
wrpRigid <- list()
clustRigid <- list()
statRigid <- list()
for(a in 1:length(regRigid)){
wrpRigid[[a]] <- antsApplyTransforms(fixed = atlas, moving = data,
transformlist = regRigid[[a]]$fwdtransforms, interpolator = c("NearestNeighbor"))
clustRigid[[a]] <- labelClusters(wrpRigid[[a]], maxThresh = 1, minClusterSize = 0, minThresh = 0.1)
statRigid[[a]] <- labelStats(wrpRigid[[a]], clustRigid[[a]])
print(statRigid[[a]])
}
In my test, I get anywhere from 7 to 11 clusters, with slightly different masses and (x,y,z) coordinates.
This was run most recently using R version 3.3.1, ANTsR version: 0.5.6, ANTsRCore version 0.3.4, but we've replicated this problem on two different Linux machines.
just a comment on the scientific result - this would suggest ( already fairly clear from the literature ) that univariate VLSM results are not very stable. of course, one can exploit this uncertainty in order to better understand the nature of the estimation problem.
worth adding some of this information to the antsRegistration
documentation in order to head off repeat discussion.
will leave this open until a contribution to the DOC is made by myself or someone else.
brian
On Thu, Mar 29, 2018 at 11:04 AM, Nick Tustison [email protected] wrote:
Please see this thread https://github.com/ANTsX/ANTs/issues/444 where these issues are discussed.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210#issuecomment-377265094, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyftSHh8knfPsPh2r_bEfQWQ43J8SIks5tjPffgaJpZM4TAbCv .
It seems like that thread (https://github.com/ANTsX/ANTs/issues/444#issuecomment-303098363) indicated changing --float to --double may help, is that correct?
If so, what about changing
https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L494
to --double.
That was also my understanding of that thread, but it doesn't seem like antsRegistration will accept a --double flag. When I try
regRigid[[i]] <- antsRegistration(fixed = atlas, moving = template, typeofTransform = c("Rigid"), "--double")
I get
ERROR: Invalid flag provided double
Massive impact on memory use and creates more maintenance overhead Due to need for converting between types ... furthermore, no guarantee it will fix the issue. Running in single threaded mode without random sampling in the metric is probably good enough. Although I have not tested that recently.
On Thu, Mar 29, 2018 at 11:43 AM John Muschelli [email protected] wrote:
It seems like that thread (ANTsX/ANTs#444 (comment) https://github.com/ANTsX/ANTs/issues/444#issuecomment-303098363) indicated changing --float to --double may help, is that correct?
If so, what about changing https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L494
to --double.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210#issuecomment-377278243, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfn2ztlIqa_SYDjAyksVVLwzrZBH2ks5tjQEngaJpZM4TAbCv .
--
brian
I'd like to test running this in single threaded mode without random sampling, but how do I specify that within ANTsR?
I think you would need dense sampling (strategy "NONE") to have completely reproducible results. With regular sampling, a perturbation is applied to the points to randomize their location within the voxel grid.
https://github.com/InsightSoftwareConsortium/ITK/blob/8a2a15f41218c925c0a89119e09419d48f83eb22/Modules/Registration/RegistrationMethodsv4/include/itkImageRegistrationMethodv4.hxx#L954-L984
Please pardon my ignorance, but I do not know enough about how ANTsR communicates with ITK (or about how ITK works) in order to implement the solutions that have been described here. As I understand it, two strategies have been proposed:
(1) use --double precision, but antsRegistration doesn't seem to take that as a flag and there may be overly negative consequences for memory use.
(2) run in single threaded mode without random sampling and use dense sampling. It's not clear to me how to control those parameters from within ANTsR.
Any advice about implementing either or both of these would be most appreciated.
antsRegistration, as defined in ANTsR, is used more like a script which simplifies the underlying antsRegistration tool in ANTs. If you need to perform your experiments within ANTsR, you might want to access the underlying program using the antsRegistration( list_of_arguments ) format. For example, to see the short and long help menus, you can type
> antsRegistration( list( '-h', 1 ) )
> antsRegistration( list( '--help', 1 ) )
You could then use double by specifying "--float 0", i.e.,
> args <- list( "-d", ... "--float", "0", ... )
> antsRegistration( args )
See here for an example. In the same file you can see how all the different transforms are defined.
I find this issue quite disturbing. Has anybody tried to check all three options to see which one resolves the problem, or whether they are all valid approaches to enhance reproducibility in an additive manner:
- double/float
- sample density (default 0.25)
- single core vs multiple core
Probably this should be part of a larger effort to enhance registration reproducibility.
Dorian
On Thu, Mar 29, 2018, 17:39 Nick Tustison [email protected] wrote:
antsRegistration, as defined in ANTsR, is used more like a script which simplifies the underlying antsRegistration tool in ANTs. If you need to perform your experiments within ANTsR, you might want to access the underlying program using the antsRegistration( list_of_arguments ) format. For example, to see the short and long help menus, you can type
antsRegistration( list( '-h', 1 ) ) antsRegistration( list( '--help', 1 ) )
You could then use double by specifying "--float 0", i.e.,
args <- list( "-d", ... "--float", "0", ... ) antsRegistration( args )
See here https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L284-L294 for an example. In the same file you can see how all the different transforms are defined.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210#issuecomment-377380669, or mute the thread https://github.com/notifications/unsubscribe-auth/AIqafd5asjfB59sBWQheJqLoP-BNvKFFks5tjVR7gaJpZM4TAbCv .
the solution:
Go into Itk ; set a fixed seed; no multi threading... should be very reproducible.
But you lose multithreading speed-up and hide the uncertainty intrinsic to the problem.
Choose your poison.
If the population statistical outputs are sensitive to tiny variations in rotation parameters, then there are other issues one should worry about.
There were a few papers that worked on voting schemes for registration. The point was to acknowledge that the solutions are uncertain and then to use that to bootstrap the estimate ie find a median solution or something along those lines.
anyway, not much here to justify new work. if you want to open an issue at ITK, then point to this:
https://github.com/InsightSoftwareConsortium/ITK/blob/ 8a2a15f41218c925c0a89119e09419d48f83eb22/Modules/Registration/ RegistrationMethodsv4/include/itkImageRegistrationMethodv4.hxx#L940-L949
which you would want to set as a fixed seed. however, there is a cost to that - how do you choose the seed?
then, you might ask - why perturb the points at all? it's b/c there is estimation bias if you sample the grid in a perfectly regular manner.
http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf
and, again, there is the fundamental computational issue: https://en.wikipedia.org/wiki/Kahan_summation_algorithm
frequent issue in scientific computing.
Thanks Brian--informative as always. I added this wiki page as an initial attempt for addressing the relevant issues. Let me know if you'd like something different.
I can confirm that using double precision is computationally impractical: I tried it for a Rigid registration and it still hadn't finished after several hours (float version takes ~20sec). Thanks everyone for an informative discussion.
should not lead to that much computation time difference - unless you are running out of memory.
brian
On Mon, Apr 2, 2018 at 10:10 AM, Dan Mirman [email protected] wrote:
I can confirm that using double precision is computationally impractical: I tried it for a Rigid registration and it still hadn't finished after several hours (float version takes ~20sec). Thanks everyone for an informative discussion.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210#issuecomment-377937407, or mute the thread https://github.com/notifications/unsubscribe-auth/AATyfgZOXHsFWPU37Ks4UzuKEMwH9PBlks5tkjFngaJpZM4TAbCv .
we can close this issue due to several recent changes in ANTs and ANTsR. you will need to define (in your environment) ANTS_RANDOM_SEED=1 (or whatever your favorite number is). e.g. my .Renviron file reads:
ANTS_RANDOM_SEED=1234
ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1
this should give you repeatable results.
ultimately, we may also allow a command line parameter to control the seed.
one note: histogram matching induced an additional unexpected source of random variability.
@ncullen93 may find this of interest as well.
In testing ANTs registration, we discovered it is not reproducible: running several times on the same inputs gives different morphisms. Implementing:
ANTS_RANDOM_SEED=1234 ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1
as suggested above, does not seem to completely resolve the issue. I see based on CPU usage that threads are not created. But it is impossible for me to check if fixing seed took effect. We are using version 2.3.1, should these variables have effect in this version?
Are there other methods to enforce reproducibility?
P.S. It is not obvious to me how randomness can help solving for brain morphism (random seed suggests it is done on purpose), but this is not the question at hand.
@moskvich412 I see that you have read a couple of the comments in this thread, specifically the last comment from @stnava dealing with ANTS_RANDOM_SEED and ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS but have you read some of the other comments pointing to some of the foundational problems with this issue that goes beyond ANTs? There are even some references pointing to discussion of these algorithmic stochastic elements and why they're important, including in the study of brain morphisms.
Yes, i did read some references, some paper on some sampling strategy, and why one by some name i can not remember which is generalization to 3D of some other name that i can not remember than uniform sampling. Other things on importance of order of summation and threads.
However, my reading does not answer the question how to make ANTs reproducible.
If humans are asked to segment same brain several times, the results will be different. Why? Because in some tough situations humans will not know how to decide, and not remember what they did last time.
If ANTs wants to mimic that aspect of human behavior, then it is likely successful. I consider this a limitation of human brain and would like ANTs to surpass humans. Why? Because when we reprocess data and get different result it is very hard to understand the cause when things are intrinsically unstable. We have to study the effect of morphism (or anything else) variability on the results, and try to understand if the observed deviation of a given reprocessing is outside of the natural variability.
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
Roman
Sent from my T-Mobile 4G LTE Device
-------- Original message -------- From: Nick Tustison [email protected] Date: 9/28/19 11:15 AM (GMT-05:00) To: ANTsX/ANTsR [email protected] Cc: Roman Fleysher [email protected], Mention [email protected] Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210)
@moskvich412https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmoskvich412&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523853370&sdata=PPIjQ64e3g2GghxMguW8aHKCy6pimGKUxVMYPo8Vvlk%3D&reserved=0 I see that you have read a couple of the comments in this thread, specifically the last comment from @stnavahttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fstnava&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=OYxzyrErwHngYrHCXNqKpqk4SCoVYRvJ%2BEaTQSVMef4%3D&reserved=0 dealing with ANTS_RANDOM_SEED and ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS but have you read some of the other comments pointing to some of the foundational problems with this issue that goes beyond ANTs? There are even some references pointing to discussion of these algorithmic stochastic elements and why they're important, including in the study of brain morphisms.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD2Z3HRSXZPY22UWIDIDQL5YKNA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD723Y4Y%23issuecomment-536198259&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=4CTnxTRrSqwerGEcuhIDU9q5JPmPobduKBJszI9ekvA%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25DASAL3YHI3N2BMCDQL5YKNANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=KMiHHbkN1CzV6CPxrtE0QSEM9h4dCnVA5NxP3a9FMBA%3D&reserved=0.
I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.
Sent from my iPhone
On Sep 28, 2019, at 11:36 AM, moskvich412 [email protected] wrote:
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal.
I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.
Sent from my iPhone
On Sep 28, 2019, at 11:36 AM, moskvich412 [email protected] wrote:
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
—
I mean the commands that you ran
Sent from my iPhone
On Sep 28, 2019, at 12:30 PM, moskvich412 <[email protected]mailto:[email protected]> wrote:
There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal.
-------- Original message -------- From: Philip Cook <[email protected]mailto:[email protected]> Date: 9/28/19 12:17 PM (GMT-05:00) To: ANTsX/ANTsR <[email protected]mailto:[email protected]> Cc: Roman Fleysher <[email protected]mailto:[email protected]>, Mention <[email protected]mailto:[email protected]> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210)
I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.
Sent from my iPhone
On Sep 28, 2019, at 11:36 AM, moskvich412 <[email protected]mailto:[email protected]> wrote:
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ.
I am also interested in having a reproducible=TRUE in antsRegistration(). Would be good to have ANTsR itself take care of the needed arguments (seed and threads) to achieve it, and perhaps return the seed number for future use.
On Sat, Sep 28, 2019, 12:36 PM Philip Cook [email protected] wrote:
I mean the commands that you ran
Sent from my iPhone
On Sep 28, 2019, at 12:30 PM, moskvich412 <[email protected] mailto:[email protected]> wrote:
There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal.
-------- Original message -------- From: Philip Cook <[email protected]<mailto: [email protected]>> Date: 9/28/19 12:17 PM (GMT-05:00) To: ANTsX/ANTsR <[email protected]mailto:[email protected]>
Cc: Roman Fleysher <[email protected]<mailto: [email protected]>>, Mention <[email protected] mailto:[email protected]> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210)
I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.
Sent from my iPhone
On Sep 28, 2019, at 11:36 AM, moskvich412 <[email protected] mailto:[email protected]> wrote:
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0>, or mute the thread< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0>.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ>.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=ACFJU7OLCTA7DZHZVPUJOVDQL6BX7A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725OVA#issuecomment-536205140, or mute the thread https://github.com/notifications/unsubscribe-auth/ACFJU7JKPPFB2L3VLDK3FT3QL6BX7ANCNFSM4EYBWCXQ .
Is the code being run in ants or ANTsR? It wasn’t entirely clear from the message. I understand it’s an issue on this repo but just wanted to make sure
On Sat, Sep 28, 2019 at 1:16 PM dorianps [email protected] wrote:
I am also interested in having a reproducible=TRUE in antsRegistration(). Would be good to have ANTsR itself take care of the needed arguments (seed and threads) to achieve it, and perhaps return the seed number for future use.
On Sat, Sep 28, 2019, 12:36 PM Philip Cook [email protected] wrote:
I mean the commands that you ran
Sent from my iPhone
On Sep 28, 2019, at 12:30 PM, moskvich412 <[email protected] mailto:[email protected]> wrote:
There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal.
-------- Original message -------- From: Philip Cook <[email protected]<mailto: [email protected]>> Date: 9/28/19 12:17 PM (GMT-05:00) To: ANTsX/ANTsR <[email protected]<mailto: [email protected]>>
Cc: Roman Fleysher <[email protected]<mailto: [email protected]>>, Mention <[email protected] mailto:[email protected]> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210)
I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.
Sent from my iPhone
On Sep 28, 2019, at 11:36 AM, moskvich412 <[email protected] mailto:[email protected]> wrote:
Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0
, or mute the thread<
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub<
https://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593
, or mute the thread<
https://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=ACFJU7OLCTA7DZHZVPUJOVDQL6BX7A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725OVA#issuecomment-536205140 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ACFJU7JKPPFB2L3VLDK3FT3QL6BX7ANCNFSM4EYBWCXQ
.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/ANTsX/ANTsR/issues/210?email_source=notifications&email_token=AAIGPLUDN4FMEEPF3CBRVM3QL6GQBA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD726IFA#issuecomment-536208404, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIGPLXOSGDESBT4GAHA4JLQL6GQBANCNFSM4EYBWCXQ .
-- Best, John
I use this bash script to compute totalTransform.nii.gz:
export ANTS_RANDOM_SEED=10 export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1
N3BiasFieldCorrection 3 input.nii.gz input.nii.gz 4 N3BiasFieldCorrection 3 template.nii.gz template.nii.gz 4
ANTS 3 -m CC[template.nii.gz,input.nii.gz,1,4] -t SyN[0.25] -r Gauss[3,0] -o prefix -i 100x90x30 --use-Histogram-Matching --number-of-affine-iterations 10000x10000x10000x10000x10000 --MI-option 32x16000
ComposeMultiTransform 3 totalTransform.nii.gz -R template.nii.gz prefixWarp.nii.gz prefixAffine.txt
The output morphism, totalTransform.nii.gz, is then applied using:
WarpImageMultiTransform 3 input.nii.gz morphedInput.nii.gz totalTransform.nii.gz -R totalTransform.nii.gz
I see that both N3BiasFieldCorrection and ANTS are running in a single thread. We are using version 2.3.1.
Comments on the choice of metrics, iterations, etc are also welcome. Maybe there is a better set.
See an example here ... looks like Mattes is not reproducible. Comments welcome.
It would be good to add Python and bash examples to above. Please contribute via pull request.
We don't recommend using the program called ANTS which is not maintained except perhaps to "get it to compile." The current tool is called antsRegistration.
python example shows similar behavior. in brief, assuming you set the environment variables appropriately:
- syn with any metric appears to be reproducible
- affine or other low-dimensional transformations with any metric other than Mattes is reproducible
not sure why Mattes no longer exhibits deterministic behavior. I agree with @cookpa that I thought we fixed this at some prior point.
@stnava I think the random sampling only happens with non-dense metrics. I think the default for SyN with Mattes uses dense sampling, so there's no jittering of the sample points.
This sounds to me that the ANTS_RANDOM_SEED variable and / or ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS is not being handled properly in the R / Py environments.
With ANTs I can run
for i in 1 2; do
antsRegistrationSyNQuick.sh -d 3 -f T_template0_BrainCerebellum.nii.gz -m 1012.nii.gz -t a -o run${i}FixSeed -n 1 -e 1234567
done
and get identical results.
@cookpa ... maybe it's a version issue. following the GitHub example https://github.com/stnava/reproducibleANTsExample, I get slightly different results using the .sh script.
export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1
export ANTS_RANDOM_SEED=1
cd src
Rscript reproducible.R
python3 reproducible.py
bash reproducible.sh
shows (for the .sh)
[1] 0.92153221 -0.01356656 0.01486456 1.05481827 -28.51814079 13.17944527
[1] 0.92480403 -0.01544587 0.02140025 1.05231535 -28.58547211 13.18869877
these are the affine parameters.
looks like a version issue; probably need to update the ants tags for ANTsR and ANTsPy ... been waiting for a stable point to do this and it seems that now(ish) may be reasonable.