atomic-write
atomic-write copied to clipboard
stackbuilders
add a call to fsync()?
I think this package would be more useful if there was a call to fsync() as well, at least optionally, as with http://hackage.haskell.org/package/safeio-0.0.5.0/docs/src/System-IO-SafeWrite.html#withOutputFile ?
@gbaz Internally, atomicWithFile, which is similar to withOutputFile, executes renameFile which calls rename Posix call. After a brief research, I found that rename is atomic. So, there will always be exactly one file after calling rename. Therefore, I'm not sure what would be the benefits of calling fsync.
@gbaz Could you please give us an explanation with a little bit more of detail? We will really appreciate it, in order to solve this issue.
The rename is indeed atomic. But the fsync is to guarantee that the data written to the new file (which is renamed) is actually written all the way to disk, rather than kept partially in cache. c.f. https://en.wikipedia.org/wiki/Sync_(Unix) or any other related documentation on fsync.
@gbaz Oh, I see. Thanks for the clarification. I'll see what we can do to solve this issue.
Note that unliftio offers durable operations in http://hackage.haskell.org/package/unliftio-0.2.12/docs/UnliftIO-IO-File.html.
Okey, after looking into it, we could durability to the functions. If we use fileSynchronize from System.Posix.Unistd as done here. However, for windows users we would need to use FlushFileBuffer from the Windows API (Win32 API) which can be found in Haskell's Win32 library. What do you think of this @CristhianMotoche and @gbaz ?
@cptrodolfox You might be interested in @lehins' comments on Windows file handling in https://github.com/fpco/unliftio/issues/50#issuecomment-555489954.
After reading through @lehins' comment https://github.com/fpco/unliftio/issues/50#issuecomment-555489954 and the documentation on FlushFileBuffer. I recon that the best way to add durability on Windows is to use the MoveFileEx with the correct flag. The function renamePath currently only has the MOVEFILE_REPLACE_EXISTING flag, we need to have additionally the MOVEFILE_WRITE_THROUGH flag.