PPI icon indicating copy to clipboard operation
PPI copied to clipboard

RT 36384: PPI won't parse source containing NUL

Open moregan opened this issue 11 years ago • 6 comments

https://rt.cpan.org/Public/Bug/Display.html?id=36384

PPI 1.215/1.216_01:

perl -WE 'open( my $fh, ">", "contains_nul.pl"); print $fh "my \$a; \0 my \$b; print 1;";'

xxd -g1 contains_nul.pl
0000000: 6d 79 20 24 61 3b 20 00 20 6d 79 20 24 62 3b 20  my $a; . my $b;
0000010: 70 72 69 6e 74 20 31 3b                          print 1;

perl contains_nul.pl
1

ppidump contains_nul.pl
Could not parse code: Encountered unexpected character '0'

moregan avatar Mar 16 '14 15:03 moregan

That's probably intentional.

:) On Mar 16, 2014 8:11 AM, "moregan" [email protected] wrote:

https://rt.cpan.org/Public/Bug/Display.html?id=36384

PPI 1.215/1.216_01:

perl -WE 'open( my $fh, ">", "contains_nul.pl"); print $fh "my $a; \0 my $b; print 1;";'

xxd -g1 contains_nul.pl 0000000: 6d 79 20 24 61 3b 20 00 20 6d 79 20 24 62 3b 20 my $a; . my $b; 0000010: 70 72 69 6e 74 20 31 3b print 1;

perl contains_nul.pl 1

ppidump contains_nul.pl Could not parse code: Encountered unexpected character '0'

Reply to this email directly or view it on GitHubhttps://github.com/adamkennedy/PPI/issues/69 .

adamkennedy avatar Mar 17 '14 20:03 adamkennedy

Are you saying that PPI should remain different from perl here? (This came up because the CPAN.pm distributed with 5.10.0 had a NUL in it.)

moregan avatar Mar 21 '14 03:03 moregan

Interesting.

PPI should handle anything that perl can run legally, so carry on.

Adam On Mar 20, 2014 8:13 PM, "moregan" [email protected] wrote:

Are you saying that PPI should remain different from perl here? (This came up because the CPAN.pm distributed with 5.10.0 had a NUL in it.)

Reply to this email directly or view it on GitHubhttps://github.com/adamkennedy/PPI/issues/69#issuecomment-38245975 .

adamkennedy avatar Mar 21 '14 19:03 adamkennedy

PPI throws the same error for the vertical tab character (ASCII 11), which 5.18 now counts as whitespace in \s

moregan avatar Mar 24 '14 05:03 moregan

Will perl itself run code with the vertical tab?

Does anyone ever actually wrote anything using it? On Mar 23, 2014 10:15 PM, "moregan" [email protected] wrote:

PPI throws the same error for the vertical tab character (ASCII 11), which 5.18 now counts as whitespace in \s

Reply to this email directly or view it on GitHubhttps://github.com/adamkennedy/PPI/issues/69#issuecomment-38412973 .

adamkennedy avatar Mar 31 '14 21:03 adamkennedy

Yes, perl will run code with VT where can be whitespace (tried 5.8.8 and 5.16.3). As of 5.18, VT is now included in \s, so at least in principle PPI should deal with it. In the 25% of CPAN that I have lying around for testing PPI with, ack finds zero VTs, which doesn't surprise me. There are a handful of CPAN modules that have NULs somewhere.

NUL, along with EOT and ESC, can appear in some places that whitespace can, but not others. I'd like to know the story there before doing anything with NUL.

moregan avatar Mar 31 '14 22:03 moregan