ExtUtils-ParseXS lacks test coverage of edge cases

[jkeenan]

I've been studying (better put: struggling with) https://github.com/Perl/perl5/issues/23846 for the past few days. I believe that CPAN distro Authen-SASL-XS has been unmaintained for so long that it may be unsalvageable. (At the very least, its needs a new maintainer). But what I want to discuss in this ticket is what the test failures discussed in that ticket show us about limitations in the coverage which ExtUtils-ParseXS's test suite provides to that distribution's code.

I was not able to get Authen-SASL-XS's Makefile.PL to complete on Debian Linux; I could not figure out what packages to install to get sasl.h. Hence, all my data is drawn from my attempts at running that distro through CPANtesters on FreeBSD-14, where I had the necessary C header files already installed. You can view my CPANtesters reports here.

What I am specifically concerned with is my attempt to build that distro against v5.43.3; that is the subject of this report. Note this output from make:

"/usr/home/jkeenan/testing/perl-5.43.3/bin/perl" "/home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/xsubpp"  -typemap '/home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/typemap' -typemap '/usr/home/jkeenan/.cpan/build/Authen-SASL-XS-1.00-1/typemap'  XS.xs > XS.xsc
Use of uninitialized value in scalar chomp at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1287, <__ANONIO__> line 2082.
Use of uninitialized value in substitution (s///) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1288, <__ANONIO__> line 2082.
Use of uninitialized value in pattern match (m//) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1288, <__ANONIO__> line 2082.
Use of uninitialized value in pattern match (m//) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1301, <__ANONIO__> line 2082.
Use of uninitialized value in pattern match (m//) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1450, <__ANONIO__> line 2082.
Use of uninitialized value in pattern match (m//) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1472, <__ANONIO__> line 2082.
Use of uninitialized value in pattern match (m//) at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1475, <__ANONIO__> line 2082.
Use of uninitialized value in string eq at /home/jkeenan/testing/perl-5.43.3/lib/5.43.3/ExtUtils/ParseXS.pm line 1523, <__ANONIO__> line 2082.
Please specify prototyping behavior for XS.xs (see perlxs manual)
mv XS.xsc XS.c

These warnings are emitted before the problems reported by @LeonT on Oct 15 in https://github.com/Perl/perl5/issues/23846#issuecomment-3406594682; I believe they are logically distinct from those problems. What concerns me is that ExtUtils-ParseXS's test suite is apparently not exercising ("covering") the statements in ParseXS.pm where those Use of uninitialized value in ... warnings are being emitted. Here are the 3 relevant subroutines with comments indicating the warnings lines.

1278 sub _maybe_skip_pod {
1279   my ExtUtils::ParseXS $self = shift;
1280 
1281   while ($self->{lastline} =~ /^=/) {
1282     while ($self->{lastline} = readline($self->{in_fh})) {
1283       last if ($self->{lastline} =~ /^=cut\s*$/);
1284     }
1285     $self->death("Error: Unterminated pod") unless defined $self->{lastline};
1286     $self->{lastline} = readline($self->{in_fh});
1287     chomp $self->{lastline};           # <-- warning #
1288     $self->{lastline} =~ s/^\s+$//;           # <-- warnings 2 #
1289   }
1290 }

#####

1296 sub _maybe_parse_typemap_block {
1297   my ExtUtils::ParseXS $self = shift;
1298 
1299   # This is special cased from the usual paragraph-handler logic
1300   # due to the HEREdoc-ish syntax.
1301   if ($self->{lastline} =~ /^TYPEMAP\s*:\s*<<\s*(?:(["'])(.+?)\1|([^\s'"]+?))\s*;?\s*$/)           # <-- warning #
1302   {
...
1321     $self->{lastline} = "";
1322   }
1323 }

#####

1418 sub fetch_para {
1419   my ExtUtils::ParseXS $self = shift;
...
1450     if ($self->{lastline} !~ /^\s*#/ # not a CPP directive           # <-- warning #
1451            # CPP directives:
1452            #   ANSI:    if ifdef ifndef elif else endif define undef
1453            #              line error pragma
1454            #   gcc:    warning include_next
1455            #   obj-c:  import
1456            #   others: ident (gcc notes that some cpps have this one)
1457         || $self->{lastline} =~ /^\#[ \t]*
1458                                   (?:
1459                                         (?:if|ifn?def|elif|else|endif|elifn?def|
1460                                            define|undef|pragma|error|
1461                                            warning|line\s+\d+|ident)
1462                                         \b
1463                                       | (?:include(?:_next)?|import)
1464                                         \s* ["<] .* [>"]
1465                                  )
1466                                 /x
...
1469       # Blank line followed by char in column 1. Start of next XSUB?
1470       last if    $self->{lastline} =~ /^\S/
1471               && @{ $self->{line} }
1472               && $self->{line}->[-1] eq "";           # <-- warning #
...
1474       # processes CPP conditionals
1475       if ($self->{lastline}           # <-- warning #
1476             =~/^#[ \t]*(if|ifn?def|elif|else|endif|elifn?def)\b/)
...
1522   pop(@{ $self->{line} }), pop(@{ $self->{line_no} })
1523     while @{ $self->{line} } && $self->{line}->[-1] eq "";           # <-- warning #
1524 
1525   return 1;
1526 }

To assess our test coverage, I created a branch forked from v5.43.3 in which I inserted print statements in dist/ExtUtils-ParseXS/lib/ExtUtils/ParseXS.pm before each of the points where uninitialized value warnings were being emitted when running make for Authen-SASL-XS on FreeBSD-14. I built a threaded perl in the way I customarily do on that platform. I then ran cd t;./perl harness -v ../dist/ExtUtils-ParseXS/t/*.t ; cd -, captured and examined the output. No warnings were emitted at any of the 7 statements indicated by # <-- warning # above.

What I infer from that research is that the .xs files we use as the corpus for ExtUtil-ParseXS's test suite don't exercise the edge cases which are revealed by our investigation into Authen-SASL-XS.

@iabyn (or anyone): How can we extend our test coverage here?