dbuenzli
really atomic file writes
The documentation says
Atomic writes.} Files are written atomically by the
functions. They create a temporary file [t] in the directory
of the file [f] to write, write the contents to [t] and
renames it to [f] on success
However this does not ensure atomicity on a POSIX filesystem. After a crash could end up with a 0-sized file (e.g. on some versions of XFS), or theoretically some other random garbage that is neither the old or the new file (if the FS flushes only the metadata about the rename, but not the contents of the file), which wouldn't be atomic anymore. There are various heuristics in certain Linux filesystems that try to detect this pattern and add missing journal flushes, but that is dependent on an implementation detail.
Just had a long discussion related to this on this PR: https://github.com/xapi-project/stdext/pull/43 and we've ended up adding an fsync between write/close/rename: write/fsync/close/rename.
(Optionally the parent dir can be fsynced too if you want to ensure that the new file will be present after a crash, but it is optional because even without it you would know that at least you have the old file if not).
This does come at a performance cost, but if the goal is to ensure the user's data is not corrupted there aren't many good alternatives. (Perhaps with_output could take an optional boolean defaulting to true on whether you want atomic writes).
See the linked PR for some articles and slides on this topic, in particular https://www.slideshare.net/nan1nan1/eat-my-data.
Unfortunately even using fsync is not enough on Mac OS X, because Mac OS X's implementation of fsync is essentially a no-op (sqlite uses an fcntl, but I'm not sure if that kind of workaround would belong in a portable library such as Bos).
