node-exporter-textfile-collector-scripts icon indicating copy to clipboard operation
node-exporter-textfile-collector-scripts copied to clipboard

sponge only writes atomically when TMPDIR is on the same filesystem

Open jbaumcloud opened this issue 5 years ago • 1 comments

Using sponge to atomically write files does not work if the environment variable TMPDIR is unset or located on a different file system. See its man page.

On default setups TMPDIR is undefined and sponge then creates a file in /tmp, which is a different file system. Using strace we can see that sponge creates a file in /tmp, attempts to rename() it to the destination, which fails with EXDEV and then moves it non-atomically.

$ echo | strace sponge foo
[…]
openat(AT_FDCWD, "/tmp/sponge.wCnFVZ", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
umask(022)                              = 077
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(3, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
read(0, "\n", 8192)                     = 1
read(0, "", 8191)                       = 0
lstat("foo", {st_mode=S_IFREG|0644, st_size=1, ...}) = 0
fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
write(3, "\n", 1)                       = 1
chmod("/tmp/sponge.wCnFVZ", 0100644)    = 0
rename("/tmp/sponge.wCnFVZ", "foo")     = -1 EXDEV (Invalid cross-device link)
openat(AT_FDCWD, "foo", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 4
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\n", 8192)                     = 1
fstat(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
read(3, "", 8192)                       = 0
close(3)                                = 0
write(4, "\n", 1)                       = 1
close(4)                                = 0
rt_sigprocmask(SIG_BLOCK, [HUP INT QUIT PIPE ALRM TERM XCPU XFSZ VTALRM PROF IO], [], 8) = 0
unlink("/tmp/sponge.wCnFVZ")            = 0
[…]

Please either mention that sponge needs a properly configured TMPDIR variable or recommend a different tool/method for atomical writing.

jbaumcloud avatar Oct 01 '19 13:10 jbaumcloud

I agree recommending to use sponge is not the best option here, even simple .... > blah.prom.part; mv blah.prom.part blah.prom is better option

laurivosandi avatar Nov 19 '21 08:11 laurivosandi