Perl
"my" variable appears to inadvertently act as "state" variable
From [email protected]
Created by [email protected]
For this script:
for (1 .. 6) { my $thing = 'foo' if /3/; $thing .= ' bar'; print "$_) thing is $thing\n"; } print "globally-scoped thing is $main::thing\n";
I would expect $thing to start out each time through the loop as undefined. Only when $_ is 3 would it be assigned 'foo'. Since $thing is a lexically-scoped variable, $main::thing should be untouched. As such, I think the output should be
1) thing is bar 2) thing is bar 3) thing is foo bar 4) thing is bar 5) thing is bar 6) thing is bar globally-scoped thing is
However, I get
1) thing is bar 2) thing is bar bar 3) thing is foo bar 4) thing is bar 5) thing is bar bar 6) thing is bar bar bar globally-scoped thing is
Somehow, $thing accumulates data with each iteration of the loop, but the data is not being stored in the globally scoped $main::thing. The problem appears to arise when the declaration "my $thing" is post-conditionalized with an "if" that evaluates to false. If this is somehow resolving to a global variable $thing, I would have expected $main::thing to contain stuff when we're done, but it does not.
Perl Info
Flags:
category=core
severity=medium
Site configuration information for perl 5.20.3:
Configured by dcmertens-perl at Sat May 21 05:52:24 EDT 2016.
Summary of my perl5 (revision 5 version 20 subversion 3) configuration:
Platform:
osname=linux, osvers=4.4.0-22-generic,
archname=x86_64-linux-thread-multi
uname='linux dcmertens-latitude-e6410 4.4.0-22-generic #40-ubuntu smp
thu may 12 22:03:46 utc 2016 x86_64 x86_64 x86_64 gnulinux '
config_args='-de
-Dprefix=/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug
-Dusemultiplicity -Dusethreads -DDEBUGGING
-Aeval:scriptdir=/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='5.3.1 20160413', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.23.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.23'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'
Locally applied patches:
Devel::PatchPerl 1.38
@INC for perl 5.20.3:
/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/lib/site_perl/5.20.3/x86_64-linux-thread-multi
/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/lib/site_perl/5.20.3
/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/lib/5.20.3/x86_64-linux-thread-multi
/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/lib/5.20.3
.
Environment for perl 5.20.3:
HOME=/home/dcmertens-perl
LANG=en_US.UTF-8
LANGUAGE=en_US
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/dcmertens-perl/perl5/perlbrew/bin:/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
PERLBREW_BASHRC_VERSION=0.74
PERLBREW_HOME=/home/dcmertens-perl/.perlbrew
PERLBREW_MANPATH=/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/man
PERLBREW_PATH=/home/dcmertens-perl/perl5/perlbrew/bin:/home/dcmertens-perl/perl5/perlbrew/perls/perl-5.20.3-thread-multi-debug/bin
PERLBREW_PERL=perl-5.20.3-thread-multi-debug
PERLBREW_ROOT=/home/dcmertens-perl/perl5/perlbrew
PERLBREW_VERSION=0.74
PERL_BADLANG (unset)
SHELL=/bin/bash
From @tonycoz
On Mon, 12 Dec 2016 07:23:24 -0800, run4flat wrote:
For this script:
for (1 .. 6) { my $thing = 'foo' if /3/; $thing .= ' bar'; print "$_) thing is $thing\n"; } print "globally-scoped thing is $main::thing\n";
I would expect $thing to start out each time through the loop as undefined. Only when $_ is 3 would it be assigned 'foo'. Since $thing is a lexically-scoped variable, $main::thing should be untouched. As such, I think the output should be
This (mis-)behaviour is well known and documented to be changable, from perlsyn:
B<NOTE:> The behaviour of a C<my>, C<state>, or C<our> modified with a statement modifier conditional or loop construct (for example, C<my $x if ...>) is B<undefined>. The value of the C<my> variable may be C<undef>, any previously assigned value, or possibly anything else. Don't rely on it. Future versions of perl might do something different from the version of perl you try it out on. Here be dragons.
It's retained for compatibility with older code which used this to emulate state variables:
my $x if 0;
but this specific usage now generates a warning.
$ ./perl -e 'my $x if 0' Deprecated use of my() in false conditional at -e line 1.
Perhaps the more general case should also produce a warning.
Tony
From [email protected]
I came upon this behavior completely unexpectedly. Would it be feasible to warn for *any* post-conditional on "my"? I suspect that the unpredictable value for a false post-conditional if-statement is almost never wanted by the programmer, except for the state emulation that you mention.
David
On Tue, Dec 13, 2016 at 11:41 PM, Tony Cook via RT < perlbug-followup@perl.org> wrote:
On Mon, 12 Dec 2016 07:23:24 -0800, run4flat wrote:
For this script:
for (1 .. 6) { my $thing = 'foo' if /3/; $thing .= ' bar'; print "$_) thing is $thing\n"; } print "globally-scoped thing is $main::thing\n";
I would expect $thing to start out each time through the loop as undefined. Only when $_ is 3 would it be assigned 'foo'. Since $thing is a lexically-scoped variable, $main::thing should be untouched. As such, I think the output should be
This (mis-)behaviour is well known and documented to be changable, from perlsyn:
B<NOTE:> The behaviour of a C<my>, C<state>, or C<our> modified with a statement modifier conditional or loop construct (for example, C<my $x if ...>) is B<undefined>. The value of the C<my> variable may be C<undef>, any previously assigned value, or possibly anything else. Don't rely on it. Future versions of perl might do something different from the version of perl you try it out on. Here be dragons.
It's retained for compatibility with older code which used this to emulate state variables:
my $x if 0;
but this specific usage now generates a warning.
$ ./perl -e 'my $x if 0' Deprecated use of my() in false conditional at -e line 1.
Perhaps the more general case should also produce a warning.
Tony
--- via perlbug: queue: perl5 status: new https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130328
-- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan
From @wolfsage
On Wed, Dec 14, 2016 at 9:08 AM, David Mertens <dcmertens.perl@gmail.com> wrote:
I came upon this behavior completely unexpectedly. Would it be feasible to warn for *any* post-conditional on "my"? I suspect that the unpredictable value for a false post-conditional if-statement is almost never wanted by the programmer, except for the state emulation that you mention.
This is something I have been working on (slowly). It would go faster if I just said "for now we will warn for the easy-to-detect general case" -- maybe that's enough?
But otherwise you can bury variable declarations deep in the op tree of a single statement and detecting them right gets a little tricky.
I think I've figured out what needs doing but I don't see having it in time for 5.26 (unless we pick the easy route for now).
-- Matthew Horsfall (alh)
From @xsawyerx
On 12/14/2016 06:05 PM, Matthew Horsfall (alh) wrote:
On Wed, Dec 14, 2016 at 9:08 AM, David Mertens <dcmertens.perl@gmail.com> wrote:
I came upon this behavior completely unexpectedly. Would it be feasible to warn for *any* post-conditional on "my"? I suspect that the unpredictable value for a false post-conditional if-statement is almost never wanted by the programmer, except for the state emulation that you mention. This is something I have been working on (slowly). It would go faster if I just said "for now we will warn for the easy-to-detect general case" -- maybe that's enough?
But otherwise you can bury variable declarations deep in the op tree of a single statement and detecting them right gets a little tricky.
I think I've figured out what needs doing but I don't see having it in time for 5.26 (unless we pick the easy route for now).
Iterations FTW?
Seriously though, nothing wrong with a start that catches some, if not all.
From @iabyn
On Thu, Dec 15, 2016 at 01:21:52PM +0100, Sawyer X wrote:
On 12/14/2016 06:05 PM, Matthew Horsfall (alh) wrote:
On Wed, Dec 14, 2016 at 9:08 AM, David Mertens <dcmertens.perl@gmail.com> wrote:
I came upon this behavior completely unexpectedly. Would it be feasible to warn for *any* post-conditional on "my"? I suspect that the unpredictable value for a false post-conditional if-statement is almost never wanted by the programmer, except for the state emulation that you mention. This is something I have been working on (slowly). It would go faster if I just said "for now we will warn for the easy-to-detect general case" -- maybe that's enough?
But otherwise you can bury variable declarations deep in the op tree of a single statement and detecting them right gets a little tricky.
I think I've figured out what needs doing but I don't see having it in time for 5.26 (unless we pick the easy route for now).
Iterations FTW?
Seriously though, nothing wrong with a start that catches some, if not all.
There's some history to 'my $x if ...' deprecations.
Back in 2004, when I were a lad, there was this thread:
http://nntp.perl.org/group/perl.perl5.porters/88428 Subject: Thoughts about my $x = $foo if $bar
which led to these commits of mine:
perl-5.8.0-3308-g76df5e8 remove C<my $x if foo> construct from core modules perl-5.8.0-3313-gdc9aa44 add deprecation warning for C<my $x if foo> and C<foo && (my $x)> perl-5.8.0-3316-gedd7382 Add tests for the C<my $x if foo> deprecation, and change the warning text
Then there was a later thread,
http://nntp.perl.org/group/perl.perl5.porters/89114 Subject: warning for my $x if 0
which decided that the general deprecation was wrong (and the implementation was wrong too: a simple 'my $x if 0' didn't warn), so the code from the earlier thread was reverted, and I added these instead:
perl-5.8.0-3389-g722969e retract 22328 and 22332: deprecation warning for my $x if foo perl-5.8.0-3391-g7921d0f add deprecation warning for my $x if 0
I can't find any discussion as to why the first set of patches was such a bad idea, but in the first thread, someone did suggest a valid use case:
$cond = ...; my $only_needed_for_cond = foo() if $cond; .... if ($cond) { ... do something with $only_needed_for_cond ...; }
Also, we may need to distinguish between
my $x if $cond; my $x = ... if $cond;
We may want to deprecate one but not the other.
-- In economics, the exam questions are the same every year. They just change the answers.
From @iabyn
On Mon, Mar 27, 2017 at 06:17:46PM +0100, Dave Mitchell wrote:
I can't find any discussion as to why the first set of patches was such a bad idea
I've now found the thread:
http://nntp.perl.org/group/perl.perl5.porters/88928 Subject: CPAN versus bleadperl
Apparently it broke everything.
-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.
From @iabyn
On Tue, Mar 28, 2017 at 01:03:02AM +0100, Dave Mitchell wrote:
On Mon, Mar 27, 2017 at 06:17:46PM +0100, Dave Mitchell wrote:
I can't find any discussion as to why the first set of patches was such a bad idea
I've now found the thread:
http​://nntp\.perl\.org/group/perl\.perl5\.porters/88928 Subject​: CPAN versus bleadperlApparently it broke everything.
Also see #122567: [PATCH] Warn when a conditional my() is used with an assignment from 2014
-- Never work with children, animals, or actors.
Hi! I know about weird and hard-to-fix Perl behavior with conditional 'my' and similar declarants. But there is at least one more situation when 'my' variable turns into 'state' variable, and without condition for 'my'. I tried to search for a bug report, but failed. This situation is actual for version 5.38 (not tested over current 5.39). It's GOTO which bypasses 'my' declaration. I know that C-compilers always warn or even error when "goto bypasses initialization of local variable", Perl does not. At least with default settings. Sample code:
use strict; sub test { my $cond = shift; my $iter = shift; if(!$cond) { goto ENDIT; } my $data; $data='true'.$iter; ENDIT: if(!$data) { $data='false'.$iter; } print "$iter: $data\n"; } test(1,1); test(0,2); test(0,3); test(1,4); test(0,5); test(0,6);
Outputs: 1: true1 2: false2 3: false2 4: true4 5: false5 6: false5
Expected: some warning or error about bypassing lexical var declaration
So, everytime when $data is assigned inside "if(!$data)" after bypassing "my $data", it becomes a 'state' variable until "my $data" is reached during some next iteration. Situation is completely the same as "my in false condition" and has the same cause, as I understand, but is somewhat more difficult to be found. And, I guess, not documented. I know all the rules like 'do not use goto....blah blah', there are some in documentation in particular, but this specific situation of using GOTO in a "C-style" within single function... would be great to have a warning or even error if lexical var is used after its declaration/initialization was bypassed by GOTO.
On Fri, Mar 15, 2024 at 06:27:19PM -0700, Yuriy Yevtukhov wrote:
I know all the rules like 'do not use goto....blah blah', there are some in documentation in particular, but this specific situation of using GOTO in a "C-style" within single function... would be great to have a warning or even error if lexical var is used after its declaration/initialization was bypassed by GOTO.
I'm not opposed to adding such a warning (or even making in it an error within the scope of a suitable 'use v5.xx'), but I suspect adding such a warning at compile time might be tricky in terms of false positives/negatives.
-- "You're so sadly neglected, and often ignored. A poor second to Belgium, When going abroad." -- Monty Python, "Finland"
