easybuild-easyblocks icon indicating copy to clipboard operation
easybuild-easyblocks copied to clipboard

fix extension filter for Perl packages

Open boegel opened this issue 2 years ago • 7 comments

(created using eb --new-pr)

The perldoc command we were using doesn't verify that all required dependenices of a Perl module are also available.

This should not get merged straight away, a couple of minor fixes will need to be added to the Perl easyconfigs, including:

  • adding the missing Pod::Parser extension;
  • changing Term::ReadLine::Gnu to Term::ReadLine (because require Term::ReadLine::Gnu triggers an error stating that it should not be used directly);
  • using 'modulename': False for the if extension (which doesn't provide a Perl module in the traditional sense, but adds support for using use if, cfr. https://metacpan.org/pod/if)

boegel avatar Mar 23 '22 20:03 boegel

Replaces #1449?

Also, the comment on the line above will need to be updated.

branfosj avatar Mar 23 '22 20:03 branfosj

Spent a little bit more time on this, made some progress but not out of the woods yet with necessary changes to Perl easyconfigs before this can be merged.

Changes to Perl-5.34.1-GCCcore-11.3.0.eb that make fewer checks fail for extensions:

--- a/easybuild/easyconfigs/p/Perl/Perl-5.34.1-GCCcore-11.3.0.eb
+++ b/easybuild/easyconfigs/p/Perl/Perl-5.34.1-GCCcore-11.3.0.eb
@@ -989,6 +989,11 @@ exts_list = [
         'source_urls': ['https://cpan.metacpan.org/authors/id/A/AS/ASPINELLI'],
         'checksums': ['409a8e0e4b1025c8e80f628f65a9778aa77ab285161406ca4a6c097b13656d0d'],
     }),
+    ('Pod::Parser', '1.65', {
+        'source_tmpl': 'Pod-Parser-%(version)s.tar.gz',
+        'source_urls': ['https://cpan.metacpan.org/authors/id/M/MA/MAREKR/'],
+        'checksums': ['3ba7bdec659416a51fe2a7e59f0883e9c6a3b21bc9d001042c1d6a32d401b28a'],
+    }),
     ('Pod::LaTeX', '0.61', {
         'source_tmpl': 'Pod-LaTeX-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/T/TJ/TJENNESS'],
@@ -1590,6 +1595,7 @@ exts_list = [
         'checksums': ['9841be5587bfb7cd1f2fe267b5e5ac04ce25e79d5cc77e5ef9a9c5abd101d7b1'],
     }),
     ('Term::ReadLine::Gnu', '1.42', {
+        'modulename': 'Term::ReadLine',
         'source_tmpl': 'Term-ReadLine-Gnu-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/H/HA/HAYASHI'],
         'checksums': ['3c5f1281da2666777af0f34de0289564e6faa823aea54f3945c74c98e95a5e73'],
@@ -1640,6 +1646,7 @@ exts_list = [
         'checksums': ['dbf7c827984951fb248907f940fd8f19f2696bc5545c0a15287e0fbe56a52308'],
     }),
     ('if', '0.0608', {
+        'modulename': False,
         'source_tmpl': 'if-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/X/XS/XSAWYERX'],
         'checksums': ['37206e10919c4d99273020008a3581bf0947d364e859b8966521c3145b4b3700'],

This is not enough though, since two checks are still failing:

  • PDF::API2:
    failing sanity check for 'PDF::API2' extension: command "perl -e 'require PDF::API2'" failed; output:
    Attempt to reload Compress/Zlib.pm aborted.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Content.pm line 11.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Content.pm line 11.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Page.pm line 15.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Page.pm line 15.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2.pm line 18.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2.pm line 18.
    Compilation failed in require at -e line 1.
    
  • IO::Compress::Bzip2
    failing sanity check for 'IO::Compress::Bzip2' extension: command "perl -e 'require IO::Compress::Bzip2'" failed; output:
    Compress::Raw::Bzip2 version 2.103 required--this is only version 2.101 at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Adapter/Bzip2.pm line 9.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Adapter/Bzip2.pm line 9.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Bzip2.pm line 11.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Bzip2.pm line 11.
    Compilation failed in require at -e line 1.
    

boegel avatar Sep 09 '22 19:09 boegel

As for the Bzip2 module it needs to be downgraded to 2.102, at the highest, to get it to require Compress::Raw::Bzip2 and Compress::Zlib versions 2.101 which are what comes internally in that Perl version.

Testing locally with that now...

And it actually solves the PDF::API2 problem too.

akesandgren avatar Sep 14 '23 06:09 akesandgren

@boegelbot please test @ generoso EB_ARGS="Perl-5.36.1-GCCcore-12.3.0.eb Perl-5.36.1-GCCcore-13.1.0.eb Perl-5.38.0-GCCcore-13.2.0.eb Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb"

boegel avatar Oct 11 '23 18:10 boegel

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=2699 EB_ARGS="Perl-5.36.1-GCCcore-12.3.0.eb Perl-5.36.1-GCCcore-13.1.0.eb Perl-5.38.0-GCCcore-13.2.0.eb Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_2699 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 11921

Test results coming soon (I hope)...

- notification for comment with ID 1758312397 processed

Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).

boegelbot avatar Oct 11 '23 18:10 boegelbot

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS Perl-5.36.1-GCCcore-12.3.0.eb
  • SUCCESS Perl-5.36.1-GCCcore-13.1.0.eb
  • SUCCESS Perl-5.38.0-GCCcore-13.2.0.eb
  • SUCCESS Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total) cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8 See https://gist.github.com/boegelbot/506f0ab448869230e58c1817484eeba9 for a full test report.

boegelbot avatar Oct 11 '23 20:10 boegelbot

https://github.com/easybuilders/easybuild-easyconfigs/pull/18789 helps a lot with making this viable to merge, but I just realized that fixing the Perl easyconfigs may not be sufficient, since this is also used in PerlModule which means we may have to fix additional easyconfigs like XML-LibXML, worker, GD, etc which also PerlModule...

I'm starting to think we should get this included with EasyBuild 5.0, since it will no doubt result in some breakage left and right (in easyconfigs we don't control) because we're stricter when testing the "import" of installed Perl modules.

boegel avatar Oct 27 '23 09:10 boegel