Perl
B::Deparse: blead incorrectly deparses __LINE__
Module: B::Deparse
Description
B::Deparse is intended to preserve semantics; i.e. given a compiled optree, it tries to generate source code that has the same effect. However, this is no longer the case in blead with expressions involving __LINE__.
Steps to Reproduce
try.pl:
use v5.38;
say __LINE__;
Code:
$ ./perl try.pl
3
$ ./perl -Ilib -MO=Deparse try.pl | perl
try.pl syntax OK
9
The deparsed version outputs 9, not 3.
Expected behavior The original script and the deparsed version should produce the same output:
$ perl try.pl
3
$ perl -MO=Deparse try.pl | perl
try.pl syntax OK
3
Perl configuration
Summary of my perl5 (revision 5 version 39 subversion 11) configuration:
Derived from: 1ab356f0f7a706c5fa6502b17f7fd0dff767a707
Platform:
osname=linux
osvers=6.5.0-10036-tuxedo
archname=x86_64-linux
uname='linux luum 6.5.0-10036-tuxedo #40 smp preempt_dynamic fri apr 26 15:33:22 utc 2024 x86_64 x86_64 x86_64 gnulinux '
config_args='-Dcc=cgcc -Dusedevel -des'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
Compiler:
cc='cgcc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='11.4.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cgcc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64
libs=-lpthread -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
libc=/lib/x86_64-linux-gnu/libc.so.6
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.35'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_LONG_DOUBLE
HAS_STRTOLD
HAS_TIMES
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_SIPHASH13
PERL_HASH_USE_SBOX32
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_DEVEL
PERL_USE_SAFE_PUTENV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
Locally applied patches:
uncommitted-changes
Built under linux
Compiled at May 8 2024 08:06:21
%ENV:
PERLBREW_BASHRC_VERSION="0.74"
PERLBREW_HOME="/home/mauke/.perlbrew"
PERLBREW_MANPATH="/home/mauke/perl5/perlbrew/perls/perl-5.38.2/man"
PERLBREW_PATH="/home/mauke/perl5/perlbrew/bin:/home/mauke/perl5/perlbrew/perls/perl-5.38.2/bin"
PERLBREW_PERL="perl-5.38.2"
PERLBREW_ROOT="/home/mauke/perl5/perlbrew"
PERLBREW_VERSION="0.94"
PERLDOC="-oman"
PERL_UNICODE="SAL"
@INC:
lib
/usr/local/lib/perl5/site_perl/5.39.11/x86_64-linux
/usr/local/lib/perl5/site_perl/5.39.11
/usr/local/lib/perl5/5.39.11/x86_64-linux
/usr/local/lib/perl5/5.39.11
On Tue, May 21, 2024 at 08:43:38PM -0700, mauke wrote:
use v5.38; say __LINE__;Code:
$ ./perl try.pl 3 $ ./perl -Ilib -MO=Deparse try.pl | perl try.pl syntax OK 9The deparsed version outputs 9, not 3.
This is the result of a deliberate recent change of mine, and I believe the change was correct. The output of './perl -Ilib -MO=Deparse' is a nine-line program, with LINE being on line 9. So the code should output 9.
If you want to preserve line numbers, then use the -l Deparse switch:
$./perl -Ilib -MO=Deparse,-l try.pl | perl
3
$
-- Never do today what you can put off till tomorrow.
It's still broken with -l.
try.pl:
use v5.38;
say
__LINE__;
Correct output:
$ perl try.pl
3
$ perl -MO=Deparse try.pl | perl
try.pl syntax OK
3
With blead and the -l switch:
$ ./perl -Ilib -MO=Deparse,-l try.pl | ./perl -Ilib
try.pl syntax OK
2
Why do you think a deparse that outputs 9 (or 2) is correct when the compiled code outputs 3?
On Wed, May 22, 2024 at 03:25:51AM -0700, mauke wrote:
Why do you think a deparse that outputs 9 (or 2) is correct when the compiled code outputs 3?
Consider part of a test script along the lines of:
{
use warnings;
my ($w, $x, $y);
local $SIG {__WARN__} = sub { $w = $_[0] };
my $l = __LINE__; $x .= $y;
like $w, qr/Use of uninitialized value.*line $l/;
}
That code works no matter what physical line the $x .= $y resides on. If you took some code along those lines, passed it through B::Deparse::coderef2text:(), then evalled it, would you expect it to still pass?
Consider this code:
printf "%s:%d: error some sort\n", __FILE__, __LINE__;
Before my change, it deparsed as:
printf "%s:%d: error some sort\n", 'foo.pl', '1';
After my change, it deparses as:
printf "%s:%d: error some sort\n", __FILE__, __LINE__;
As another example, given the following:
$ cat foo.pl
printf "Executing %s\n", __FILE__;
$ perl -MO=Deparse foo.pl > bar.pl
what would you expect the following to output:
$ perl ./foo.pl
$ perl ./bar.pl
I would expect :
Executing foo.pl
Executing bar.pl
-- Modern art: "That's easy, I could have done that!" "Ah, but you didn't!"
Consider part of a test script along the lines of:
{ use warnings; my ($w, $x, $y); local $SIG {__WARN__} = sub { $w = $_[0] }; my $l = __LINE__; $x .= $y; like $w, qr/Use of uninitialized value.*line $l/; }That code works no matter what physical line the $x .= $y resides on. If you took some code along those lines, passed it through B::Deparse::coderef2text:(), then evalled it, would you expect it to still pass?
Not necessarily, but the funny thing is that this case is mostly unaffected by the change in blead.
catch.pl:
use Test::More;
{
use warnings;
my ($w, $x, $y);
local $SIG {__WARN__} = sub { $w = $_[0] };
my $l = __LINE__; $x .= $y;
like $w, qr/Use of uninitialized value.*line $l/;
}
done_testing;
Before (v5.38.2):
$ perl -MO=Deparse catch.pl | perl
catch.pl syntax OK
not ok 1
# Failed test at - line 11.
# 'Use of uninitialized value $y in concatenation (.) or string at - line 10.
# '
# doesn't match '(?^:Use of uninitialized value.*line 6)'
1..1
# Looks like you failed 1 test of 1.
After (blead):
$ ./perl -Ilib -MO=Deparse catch.pl | ./perl -Ilib
catch.pl syntax OK
not ok 1
# Failed test at - line 11.
# 'Use of uninitialized value $y in concatenation (.) or string at - line 10.
# '
# doesn't match '(?^:Use of uninitialized value.*line 9)'
1..1
# Looks like you failed 1 test of 1.
On the other hand, with the -l flag both cases pass (v5.38.2 and blead):
$ perl -MO=Deparse,-l catch.pl | perl
catch.pl syntax OK
ok 1
1..1
$ ./perl -Ilib -MO=Deparse,-l catch.pl | ./perl -Ilib
catch.pl syntax OK
ok 1
1..1
As another example, given the following:
$ cat foo.pl printf "Executing %s\n", __FILE__; $ perl -MO=Deparse foo.pl > bar.plwhat would you expect the following to output:
$ perl ./foo.pl $ perl ./bar.pl
This one I'm ambivalent about. But given how constants (and inlining) work in Perl, I tend towards
Executing ./foo.pl
Executing ./foo.pl
On Wed, May 22, 2024 at 04:07:47AM -0700, mauke wrote:
Not necessarily, but the funny thing is that this case is mostly unaffected by the change in blead.
Ok, a slightly bad example. It could be tweaked to demonstrate my point. But my original motivation for this change was that in the perl distribution, you can do:
$ cd t
$ ./TEST -deparse
which runs all the 2800 test scripts, but first puts them through a round trip through the deparser. It's a useful feature that allows the deparser to be tested across a wide body of code, and thus breakages are more likely to be spotted.
A common breakage was scripts which used LINE in some fashion, and after the round trip, one or more tests failed. My change fixed some of those.
As another example, given the following:
$ cat foo.pl printf "Executing %s\n", __FILE__; $ perl -MO=Deparse foo.pl > bar.plwhat would you expect the following to output:
$ perl ./foo.pl $ perl ./bar.plThis one I'm ambivalent about. But given how constants (and inlining) work in Perl, I tend towards
Executing ./foo.pl Executing ./foo.pl
Well I'd expect the second one to output ./bar.pl, and would find it perverse if instead it output ./foo.pl.
Consider a program which assembles some code text and evals it. A compiled sub within that eval is then stringified with B::Deparse::coderef2text(&f), and the result written to a new file (along with some other stuff); perhaps by some sort of serialisation library.
When executing that new file, foo.pl say, would you expect it to start reporting warnings or errors as coming from foo.pl or from 'eval17', assuming there was some sort of warn function which used FILE etc?
-- You live and learn (although usually you just live).