BIDSonym
BIDSonym copied to clipboard
PeerHerholz
Add some tests/ci
Would be nice if there was at least some regression testing automatically ran by ci
Basic default tests are now running here. @yarikoptic what's your take on further tests? For example I thought about including defaced versions of the same image (defaced using the different algorithms) and running all defacing algorithms, comparing the overlap of defaced images using e.g. dice or jaccard. However, I'm not sure if that would make sense/would be useful!?
Even a smoke test run on some small (2 subjects) dataset and sweeping through various defacers and making sure that output is produced as expected -- would be super useful imho. Could probably done independently on github actions on a cron job using bidsonym container available. Adding comparison between results would be next step indeed.
Hi @yarikoptic do you have any examples that go in this direction? I would try to implement it now.
Why not to just take some anatomical (crap -- I didn't collect any as part of http://datasets.datalad.org/?dir=/dbic/QA/sub-sid000143/ses-20170406), e.g. from good old haxby2001 dataset (first 5 subjects have anats): http://data.pymvpa.org/datasets/haxby2001/.git , and then running a defacer and verify that in one target roi (face, could be just a known pixel + radius) - all voxels are zeroed out, and then that voxels in another roi (non-face) are all 100% original. Instead of ROIs you could hand-craft target "minimal" and "maximal" masks, and use "minimal" to verify that all those are 0s, and "non-maximal" - that all are original.
Ok, I see. Given that the different algorithms and their masks, defining one face/no-face voxel mask for all would be difficult, except we use masks where we can be rather certain that it's removed in all approaches. However, especially mri_deface is something else here as it mainly removes the eyes but e.g. keeps the nose.