read from append write only file handle breaks the following writes

[xworld21]

Description Opening a file in append write only mode (>>) and attepting to read from it puts the file handle in some kind of invalid state. Following writes will have the intended effect, i.e. append bytes to the file, but return errors.

This behaviour started with 5.38 and broke the tests in Win32::LongPath https://github.com/rdboisvert/Win32-LongPath/issues/14.

cc @shawnlaffan

Steps to Reproduce

open(my $fh, '>>', 'append') or die 'cannot open';
my $line = readline $fh;
print $fh 'append only' or die 'cannot write'; # perl >=5.38 will die here
close $fh or die 'cannot close';

Expected behavior The code should run successfully as it did before.

Note that using sysread and syswrite does not trigger the issue.

Perl configuration Observed on multiple platforms (Linux, Windows, macOS), for instance:

Summary of my perl5 (revision 5 version 38 subversion 2) configuration:
   
  Platform:
    osname=darwin
    osvers=24.2.0
    archname=darwin-thread-multi-2level
    uname='nixpkgs'
    config_args='-de -Dprefix=/nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2 -Dman1dir=/nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/share/man/man1 -Dman3dir=/nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/share/man/man3 -Dcc=cc -Duseshrplib -Uinstallusrbinperl -Dinstallstyle=lib/perl5 -Dlocincpth=/no-such-path/include -Dloclibpth=/no-such-path/lib -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=11.3 -DNO_POSIX_2008_LOCALE -fno-strict-aliasing -pipe -fstack-protector-strong -I/no-such-path/include'
    optimize='-O3'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=11.3 -DNO_POSIX_2008_LOCALE -fno-strict-aliasing -pipe -fstack-protector-strong -I/no-such-path/include'
    ccversion=''
    gccversion='Clang 19.1.7'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=11.3 -fstack-protector-strong -L/no-such-path/lib'
    libpth= 
    libs=-lcrypt
    perllibs=-lcrypt
    libc=
    so=dylib
    useshrplib=true
    libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=11.3 -bundle -undefined dynamic_lookup -L/no-such-path/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under darwin
  Compiled at Nov 28 2023 14:04:47
  %ENV:
    PERL5LIB="/nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl"
  @INC:
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl/5.38.2/darwin-thread-multi-2level
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl/5.38.2
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl/5.38.2/darwin-thread-multi-2level
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/site_perl/5.38.2
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/5.38.2/darwin-thread-multi-2level
    /nix/store/waf13fynkk30napkvzljabgsqg0fjck6-perl-5.38.2/lib/perl5/5.38.2

[jkeenan]

Placing the OP's code sample into a file, I bisected with the following invocation:

perl Porting/bisect.pl \
--start=v5.36.0 \
--end=v5.38.0 \
-- "./perl -Ilib $P5P_DIR/gh-23026.pl"

The change of behavior ("first bad commit") was observed at

commit 80c1f1e45e8ef8c27d170fae7ade41971fe20218
Author:     Tony Cook <[email protected]>
AuthorDate: Tue Aug 16 15:52:04 2022 +1000
Commit:     Tony Cook <[email protected]>
CommitDate: Wed Aug 31 10:51:09 2022 +1000

    only clear the stream error state in readline() for glob()
    

@tonycoz, can you take a look? Thanks.

[tonycoz]

Calling read() would also set the error flag on the stream, and always has.

$ ~/perl/5.10.0-debug/bin/perl -e 'open my $fh, ">>", "append"; read $fh, $buf, 10; print $fh "append only" or die "append $!"; close $fh or die "close $!"'
append Bad file descriptor at -e line 1.

The change @jkeenan listed simply prevented readline() from clearing the error state after an error.

[xworld21]

Following up on this, thanks for the clarifications. So this is intended behaviour after all. I suppose this issue should be closed, then?

I have submitted a patch to Win32::LongPath adding clearerr in the relevant test to mimic the pre-5.38 behaviour.

[ap]

Do I understand correctly that this is the same underlying issue as #22883?

[tonycoz]

Do I understand correctly that this is the same underlying issue as #22883?

Yes, the read failure sets the error flag on the stream (which read() has always done, but readline() didn't), which causes close() to fail.

[ap]

We will not address this during this cycle. But it ought to be addressed during next cycle.

[toddr]

I'm fine with whatever decision is made for what p5p decides to ship. I do want to be clear that at this point we plan to reverse 8d6c0b5d15ae392931618429f028a8841d7bc873 and 80c1f1e45e8ef8c27d170fae7ade41971fe20218 in the Perl 5.42.0 we (cPanel) will ship. It makes for a very problematic interface for us. These errors very much feel like reporting failures unrelated to the existing action.

From what I can tell with the existing behavior, once I have a breaking error on the file handle, regardless of whether I fix it, I have to re-open the file handle to clear the error. If this is true, this feels very awkward to me.

Also embedding system perl (5.36 in my case) vs what we'll get if we switch to the pending 5.42 behavior. In both cases, the temp file is written:

$>/usr/bin/perl -w -e 'open my $fh, ">", "/tmp/tsm_test.txt" or die "$!"; my $line; eval { $line = <$fh>; } or print "Failed to read: $!\n"; my $res = print $fh "Hello\n"; print "Write result: $res\n";'
Filehandle $fh opened only for output at -e line 1.
Failed to read: Bad file descriptor
Write result: 1

$>perl542 -w -e 'open my $fh, ">", "/tmp/tsm_test.txt" or die "$!"; my $line; eval { $line = <$fh>; } or print "Failed to read: $!\n"; my $res = print $fh "Hello\n"; print "Write result: $res\n";'
Filehandle $fh opened only for output at -e line 1.
Failed to read: Bad file descriptor
Use of uninitialized value $res in concatenation (.) or string at -e line 1.
Write result:

[haarg]

You can clear the error flag on a handle using the ->clearerr method.