CCF icon indicating copy to clipboard operation
CCF copied to clipboard

Add `fsync()` of _directories_ when renaming files to `.committed`

Open eddyashton opened this issue 6 months ago • 0 comments

While reading about atomic rename semantics as part of #7029, we've surfaced something further that we may want to do to ensure the renames succeed, with the contents of the newly created file. Specifically, if we don't fsync() the parent directory, it is technically possible for the rename (especially around restarts) to access a version of the directory metadata that doesn't know the file exists. So we should, when dealing with .committed files:

  1. Before closeing a .committed file, fsync() that file
  2. After close(), call fsync() on the parent directory, to minimise loss window, and be sure it's known before an incoming rename() call
  3. After rename(), call fsync() on the destination directory, to minimise the window where the rename could be lost

This probably requires shifting to direct fopen/fwrite calls, so we have file-descriptor access for fsync(), and dropping the current use of std::filesystem/std::ofstream in many places.

eddyashton avatar Jun 02 '25 14:06 eddyashton