HDF5 stability issues

[gaow]

Perhaps it is worth noticing that HDF5 does not have error recovery mechanism and there are quite a few complaints if you google "corruption hdf5". Cautions have to be used here if we decide to use HDF5 as the replacement.

[gaow]

http://cyrille.rossant.net/moving-away-hdf5/

We should be good if we can avoid re-writing the HDF5 database once it is build -- but this is not what the current SQLite implementation does. In the current SQLite version I do experience corrupted database from time to time but it can usually be recovered with a line of Linux command plus hours of wait.