charm
charm copied to clipboard
charmplusplus
Update AMPI's version of ROMIO
Original issue: https://charm.cs.illinois.edu/redmine/issues/952
ROMIO is no longer released/packaged separately from MPICH, but it still has its own separate build system and its structure is still very similar to the version (ROMIO-1.2.4) that we package with AMPI.
ROMIO-1.2.4 was released in May, 2002, while the latest version of ROMIO that ships with MPICH-3.2 is from March 2008. It makes use of some MPI-2.2 features (mostly datatype related ones) and should have improved performance.
Original date: 2017-01-12 05:23:39
Need to implement generalized requests, datatypes, and anything else it uses that we don't currently support.
Original date: 2017-06-10 22:06:19
Note that the version that is currently integrated in AMPI is based (mostly) on MPICH2 0.91. Integrating ROMIO 1.2.5 from MPICH2 0.92 is already not trivial, since the configure format changed.
For reference, old mpich versions with ROMIO are here: ftp://ftp.mcs.anl.gov/pub/mpi/old
The closest match I have found to our current ROMIO is: ftp://ftp.mcs.anl.gov/pub/mpi/old/mpich-1.2.5-1a.tar.gz
Official MPICH git repository (includes ROMIO) is here: https://github.com/pmodels/mpich
Despite the last change note in the README file being from 2008, ROMIO still receives regular patches in the git repo.
Original date: 2017-06-12 21:44:32
I'm preparing a series of patches to update our ROMIO to 1.2.5-1a (first patch here: ~~https://charm.cs.illinois.edu/gerrit/#/c/2665/~~ https://github.com/UIUC-PPL/charm/commit/36e9e76eedfff68b23bed72452530181aab1e3f3 ).
Currently, 15 out of 16 ROMIO tests pass with AMPI (with all patches applied):
ROMIO tests working:
- simple, async, i_noncontig, misc, shared_fp, error, status, file_info, noncontig, coll_test, noncontig_coll, noncontig_coll2, split_coll, excl, atomicity
ROMIO tests NOT working (1):
- psimple (needs PMPI support)
Original date: 2017-06-19 04:58:48
With the MPI_Comm_dup patch (~~https://charm.cs.illinois.edu/gerrit/#/c/2706/~~ https://github.com/UIUC-PPL/charm/commit/33836785127d8b8b7cfd582d9b573370afa54d2a), all ROMIO tests (except the one requiring PMPI) pass.
Original date: 2017-07-13 03:03:15
Marking "In Progress" to reflect the fact that we've updated to a more recent version, though not the most recent version.
Original date: 2017-08-29 17:27:26
Update our ROMIO to v1.2.6, which can be built as a shared object / dynamic library: https://charm.cs.illinois.edu/gerrit/#/c/2940/
Original date: 2017-08-29 17:28:55
Note that the patch mentioned above (https://charm.cs.illinois.edu/gerrit/#/c/2940/) does not enable compilation as a shared library yet. Some manual changes to the Makefile are necessary.
Original date: 2017-12-19 20:48:26
We'd still like this to be in 6.9.0 if possible. I don't expect 6.9.0 to be released in the next 3-4 weeks.
Original date: 2019-02-12 17:39:45
~~https://charm.cs.illinois.edu/gerrit/c/charm/+/4933~~ https://github.com/UIUC-PPL/charm/commit/b25d36ce2fb19985ba0494481c2a1e4afe2c7d53
Original date: 2019-02-12 18:00:42
The patch above updates ROMIO to the version shipped with mpich2-1.4.1p1. For newer versions of ROMIO, we would need support for the external32 data representation, including an implementation of MPI_Pack_external and MPI_Unpack_external). With such support, we could run all ROMIO versions up to 3.3 at least.
Original date: 2019-02-12 18:12:29
I think we can implement the MPI_Pack_external() and MPI_Unpack_external() routines as calls to MPI_Pack() and MPI_Unpack(), since we don't insert any kind of header inside our MPI_Pack and MPI_Unpack routines (it just copies the bytes as they are). On little-endian systems though we should convert to big-endian for external32.
Original date: 2019-02-12 18:23:10
Apart from the endianness conversion, wouldn't we also need to adjust the size of data types to the external32 format?
Original date: 2019-02-12 18:33:26
Yeah, that too. Looking into it more, it might be worth us checking to see if HDF5 uses MPI_File_set_view with external32 or something. If not, we could implement the external32 routines as runtime aborts or something
Original date: 2019-02-12 18:47:35
From what I could see, hdf5-1.10.1 only calls MPI_File_set_view with the "native" data format, which should be fine. Independent of that, having fake external32 routines can be useful in any case. For the long term, we could look at copying mpich's external32 implementation (which is a few thousand lines of code :-S).
Original date: 2019-02-15 20:29:58
Gerrit patch 4933 to upgrade us to mpich2-1.4.1p1 is merged. I'll leave this issue open since it's not the newest version (see above discussion).
WIP patch to update ROMIO to 3.2.1: https://charm.cs.illinois.edu/gerrit/c/charm/+/5151
ROMIO has been upgraded to 3.2.1 in PR #2141. Newer versions of ROMIO (3.3+) consist of two libraries, libmpio and libmpl, and will require more changes to our build system.
Since we now embed ROMIO's objects into the AMPI library and no longer use its own .a/.so etc, are the two separate libraries still a major blocker on updating ROMIO further?