ANTsX
Allow compression to be disabled on save
As data files increase in size compression can be a very significant slow down in processing, it would be great if it could be controlled. In the short term shutting it off would be great, maybe a fast and parallel compression could be done at a later time.
https://github.com/ANTsX/ANTs/blob/5b7c090b0dd8f911b013b374a06177fc0ac5c60e/Utilities/ReadWriteData.h#L525
I think an environment variable would be the simplest answer. I propose looking for "ANTS_COMPRESSED_WRITE" with value of 1 meaning use compression,or 0 meaning do not use compression. Invalid values being treated as true seems fine.
Modification will likely have to be more extensive than you might first guess. The default output filename is ".nii.gz" and as far as I can tell, that's not handled by SetUseCompression in the writer, but rather by implicit file naming.
This will affect h5 and MINC files at least, which have internal transparent compression.
Thanks @gdevenyi . Hey @jamesjcook , can you remind me what the size of images you're dealing with and some of the current compression times that you're seeing? I know it was something pretty large and I wanted to be explicit about how big of a problem this is for your data.
Slightly related, the https://github.com/zlib-ng/zlib-ng project I have been following closely and they have a substantially more performant zlib implementation which does work with ITK/ANTs (light testing by me).
For a warp at ~ 800x1300x600 it is taking approximately 30 minutes to save. After a registration(~8hrs) for both warps it was ~1hr. I'm also hoping to shorten our time to apply transform with this, as reading compressed takes a similar amount of time.
I have larger more problematic datasets at 7000x10000x6000, which I've not run through ants yet, but I have saved using compression, which took on the order of hours.
Adding to this it would be nice to be able to use formats other than NIFTI for warps. Edit: already requested in https://github.com/ANTsX/ANTs/issues/1092
