Simulations are slowed down considerably when compressing output
The simulation time is extended a couple of seconds when Tags.DO_FILE_COMPRESSION=True is in the Settings. This slows down even more other simulations modules such as MCXAdapterReflectance.
A first attempt to alleviating this is to save the HDF5 file only once and/or saving results with half precision.
I think we should make available other compression filters in SIMPA. At the moment we are using gzip as default. However, other filters such as lzf would considerably speed up the compression. A preliminary experiment with the minimal_optical_simulation example in the repo generates something like:
- using
lzf:time=0.98seconds resulting in a177.6MB.hdf5file - using
gzip:time=4.82seconds resulting in a158.5MB.hdf5file
More comparisons can be found here http://www.h5py.org/lzf/. I think we should open these options to the users @jgroehl @kdreher.