mpifileutils icon indicating copy to clipboard operation
mpifileutils copied to clipboard

dcmp: add checkpointing with file batches

Open adammoody opened this issue 5 years ago • 0 comments

For long running dcmp jobs, it would be useful to checkpoint the state.

It may be possible to do this by batching files, as with dsync. In this case, we'd write out the file list for both the source and destination. We'd have to write out the current status of any checks (the list of matching files, and perhaps the check itself). We'd also need to record a marker of how far we are along in the source/destination lists.

We'd want to take a path from the user where this checkpoint data could be saved, and then take that path again on a restart.

adammoody avatar Oct 03 '19 19:10 adammoody