coreutils icon indicating copy to clipboard operation
coreutils copied to clipboard

Should uutils-coreutils respect stdbuf buffering options?

Open Ecordonnier opened this issue 6 months ago • 0 comments

Context: while trying to implement a regression-test for https://github.com/uutils/coreutils/issues/6591 I noticed that testing stdbuf with some GNU coreutils utilities work, but fails with uutils coreutils utilities.

  • stdbuf -o works by changing the buffering option. Specifically by preloading libstdbuf.so which contains an overriden version of setvbuf. This function takes precedence over the normal setvbuf function of the libc library.

  • when C programs call IO functions like printf, setvbuf() gets called

  • when Rust programs call IO functions like print or println, setvbuf() does not get called, and thus stdbuf -o has no effect (at least this is what my testing is showing. See also this explanation

  • in theory, the buffering behavior of C programs could be emulated in rust e.g. by reading the value of the environment variable "_STDBUF_O" which is set by stdbuf, or by somehow calling setvbuf for IO.

Should this emulation be implemented in uutils-coreutils? I checked the POSIX specification, and have not found an explicit specification of the buffering behavior for e.g. "cut", and it seems to be rather part of the C standard. So as far as I understand uutils-coreutils could be POSIX compliant without this behavior, but it would be a small difference with GNU coreutils.

Details of the test: Process 1 doing:

#!/bin/bash

set -x

COREUTILS=./target/debug/coreutils
rm -f log_file2
touch log_file2
$COREUTILS tail -f log_file2 | $COREUTILS stdbuf -o1000000 $COREUTILS cut -d " " -f 1
#tail -f log_file2 | stdbuf -o1000000 cut -d " " -f 1

Process 2 doing:

#!/bin/bash

echo "A 1" >> log_file2

If stdbuf has an effect, you should see nothing in the stdout of process 1 (happens with GNU coreutils). If stdbuf has no effect, you should see "A" in the stdout of process 1 (happens with uutils coreutils).

Ecordonnier avatar May 21 '25 21:05 Ecordonnier