Evaluation: Speedup of Close

[ax3l]

@psychocoderHPC found in a ~~past evaluation~~ paper (ref?), that the time for an H5Fclose can be reduced by about a factor 3 if the file-consistency check (H5F.c) is skipped in the HDF5 implementation.

Consequences for MPI-I/O output could be evaluated. It might be worth to consider adding a hook/option in the HDF5 API and to propose it for upcoming versions of the library.

There are also various close degrees available such as H5F_CLOSE_WEAK while the default for MPI-I/O is H5F_CLOSE_SEMI.

[psychocoderHPC]

Tuning HDF5 for Lustre File Systems (2010)

 ... However, when operating
in a parallel application, this operation resulted in a call
to MPI_File_set_size, which currently has very poor
performance characteristics on Lustre file systems. 1 Because
an HDF5 file’s size is not required to be accurately set until
the file is closed, this operation was removed from H5Fflush
and added to the code for closing a file.

[ax3l]

And the good old Parallel HDF5 Hints