perl5 binaries mismatched again

Migrated from rt.perl.org#133440 (status was 'open')

Searchable as RT133440$

Aug 10 '18 18:08 p5pRT

From @doughera88

On Fri, Aug 10, 2018 at 11:24:20AM -0700, (via RT) wrote:

# New Ticket Created by
# Please include the string: [perl #133440] # in the subject line of all future correspondence about this issue. # <URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133440 >

This is a bug report for perl from frederik@ofb.net, generated with the help of perlbug 1.41 running under perl 5.28.0.

----------------------------------------------------------------- [Please describe your issue here]

I was able to find that I already reported this bug, sorry:

https://rt.perl.org/Public/Bug/Display.html?id=130718

The problem is that when running Perl code I see error messages containing the text "loadable library and perl binaries are mismatched".

Am I an atypical user for having basic stuff like Compress::Raw::Zlib installed "locally" (i.e. in my home directory)? (I'm not sure how it got there)

I'm not trying to be facetious, I just have a confusing situation and I don't know why other people are not also finding it as confusing as I do. Maybe there is a simple answer.

The immediate problem is that you have version-specific stuff stored in a non-version-specific directory. That is, looking at your @INC, you have "locally" installed modules in

/home/frederik/\.local/lib/perl5/x86\_64\-linux\-thread\-multi

Perl's configuration defaults are to store version-specific stuff in version-specific directories. Lots more details are in the INSTALL file, under the section "Coexistence with earlier versions of perl 5".

I'm not sure why you have Compress::Raw::Zlib installed locally, since it has been bundled with perl since version 5.9.4. Of course you are certainly welcome to install different versions than those bundled with perl, but if you are going to do so, and if you're going to store them in a non-version-specific directory, you'll need to recompile them when you upgrade to a new major version. Upgrade time is also a good time to re-visit the question of whether you want to continue overriding the standard version with your own local version.

Hope that helps a little,

-- Andy Dougherty doughera@lafayette.edu

Aug 10 '18 20:08 p5pRT

The RT System itself - Status changed from 'new' to 'open'

Aug 10 '18 20:08 p5pRT

From [email protected]

The immediate problem is that you have version-specific stuff stored in a non-version-specific directory. That is, looking at your @INC, you have "locally" installed modules in
/home/frederik/\.local/lib/perl5/x86\_64\-linux\-thread\-multi
Perl's configuration defaults are to store version-specific stuff in version-specific directories. Lots more details are in the INSTALL file, under the section "Coexistence with earlier versions of perl 5".

I'm not sure why you have Compress::Raw::Zlib installed locally, since it has been bundled with perl since version 5.9.4. Of course you are certainly welcome to install different versions than those bundled with perl, but if you are going to do so, and if you're going to store them in a non-version-specific directory, you'll need to recompile them when you upgrade to a new major version. Upgrade time is also a good time to re-visit the question of whether you want to continue overriding the standard version with your own local version.

Hope that helps a little,

Thank you, it does. I already know a bit about the technical reason for the problem. Perhaps I installed something that depended on Compress::Raw::Zlib over 10 years ago, then that module ended up in a list of modules that I think I need to run various personal scripts and this is why I have it installed locally. What's this about INSTALL and version-specific directories? Did I configure things wrong when I set up cpanm?

I guess I'm wondering why more people aren't coming to you with questions about this. And why not change the error message to point to a manual page which explains what to do. By "discoverability" I meant, a way for users to understand what they're seeing by reading the documentation that logically presents itself, for instance, if a module author says "you can install my module using the cpan tool", then two years later the user gets this error message, what is he expected to do next, given lack of omniscience - i.e. only knowing what he had to know to install cpan, and knowing the text of the error message.

Reading what you wrote, I downloaded 'perl' and looked at INSTALL, at all the lines containing 'version.*directory' (I didn't have time to read all 3000 lines yet), and it didn't really answer my question. But this is not something I think most users would be expected to do; what happens is you get Perl from your distro and then you see there is a module in CPAN but not in the distro, so you install it locally using cpanm, etc. Then things break.

I would like you to say, "Frederick, here is where you went wrong, you made a mistake that other users are not making when you did X". Because that would explain why you haven't been getting other bug reports about this. Lacking such an explanation, I'm kind of fearful that the answer is that Perl has just become a tool of an older generation of power users, many young programmers these days haven't even heard of it, they use bland corporate languages with fewer "gotchas"... (I'm only guessing here, because I don't use them myself...) But maybe that's off-topic.

By the way I think I set up cpanm entirely by adding lines from `perl -Mlocal::lib=.local` to my shell profile.

Best wishes,

Frederick

Aug 11 '18 04:08 p5pRT

From @bulk88

On Fri, 10 Aug 2018 11:24:20 -0700, frederik@ofb.net wrote:

Maybe I have a bad memory, but it came up again and I wrote a new report before I remembered the old one. Here it is:

Every time I upgrade my system, I run into this problem. But I don't upgrade often enough to remember what the solution is.

The problem is that when running Perl code I see error messages containing the text "loadable library and perl binaries are mismatched".

These messages don't point to a remedy, or even a specific module; instead they give the name of a C source file in a module. Googling is not very helpful.

https://perldoc.perl.org/perldiag.html#List-form-of-piped-open-not-implemented

%s: loadable library and perl binaries are mismatched (got handshake key %p, needed %p) (P) A dynamic loading library .so or .dll was being loaded into the process that was built against a different build of perl than the said library was compiled against. Reinstalling the XS module will likely fix this error.

-- bulk88 ~ bulk88 at hotmail.com

Aug 12 '18 19:08 p5pRT

From @Leont

On Sat, Aug 11, 2018 at 6:23 AM, <frederik@ofb.net> wrote:

I guess I'm wondering why more people aren't coming to you with questions about this. And why not change the error message to point to a manual page which explains what to do. By "discoverability" I meant, a way for users to understand what they're seeing by reading the documentation that logically presents itself, for instance, if a module author says "you can install my module using the cpan tool", then two years later the user gets this error message, what is he expected to do next, given lack of omniscience - i.e. only knowing what he had to know to install cpan, and knowing the text of the error message.

Reading what you wrote, I downloaded 'perl' and looked at INSTALL, at all the lines containing 'version.*directory' (I didn't have time to read all 3000 lines yet), and it didn't really answer my question. But this is not something I think most users would be expected to do; what happens is you get Perl from your distro and then you see there is a module in CPAN but not in the distro, so you install it locally using cpanm, etc. Then things break.

I would like you to say, "Frederick, here is where you went wrong, you made a mistake that other users are not making when you did X". Because that would explain why you haven't been getting other bug reports about this. Lacking such an explanation, I'm kind of fearful that the answer is that Perl has just become a tool of an older generation of power users, many young programmers these days haven't even heard of it, they use bland corporate languages with fewer "gotchas"... (I'm only guessing here, because I don't use them myself...) But maybe that's off-topic.

By the way I think I set up cpanm entirely by adding lines from `perl -Mlocal::lib=.local` to my shell profile.

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

Leon

Aug 13 '18 21:08 p5pRT

From @iabyn

On Mon, Aug 13, 2018 at 11:56:26PM +0200, Leon Timmermans wrote:

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

From a quick perusal of local::lib's documentation, it appears that it doesn't use version-specific paths under the local directory, which on the face of it seems like a design flaw.

-- If life gives you lemons, you'll probably develop a citric acid allergy.

Aug 14 '18 08:08 p5pRT

From @eserte

Dana Tue, 14 Aug 2018 01:42:09 -0700, davem reče:

On Mon, Aug 13, 2018 at 11:56:26PM +0200, Leon Timmermans wrote:

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

From a quick perusal of local::lib's documentation, it appears that it doesn't use version-specific paths under the local directory, which on the face of it seems like a design flaw.

Interestingly, local::lib creates version-specific paths on first-time initialization. But I think the problem is that by using ExtUtils::MakeMaker's INSTALL_BASE no installation to version-specific paths happens, so the design flaw is probably there. (Did not check Module::Build's --install_base)

Aug 14 '18 18:08 p5pRT

From @Leont

On Tue, Aug 14, 2018 at 10:41 AM, Dave Mitchell <davem@iabyn.com> wrote:

On Mon, Aug 13, 2018 at 11:56:26PM +0200, Leon Timmermans wrote:

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

From a quick perusal of local::lib's documentation, it appears that it doesn't use version-specific paths under the local directory, which on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least it doesn't for the common scenario where one has multiple (incompatible) perls and at login time one doesn't know yet which to use when. The only primitive we provide (PERL5LIB) does "prepend these dirs to @INC". I can sort of imagine a tool that does what you suggest, but the additional complexity and fragility sound uncomfortable.

It's a real problem, but I don't think it can be solved on the local::lib side alone.

Leon

Aug 14 '18 23:08 p5pRT

From [email protected]

On Wed, Aug 15, 2018 at 01:19:16AM +0200, Leon Timmermans wrote:

On Tue, Aug 14, 2018 at 10:41 AM, Dave Mitchell <davem@iabyn.com> wrote:

On Mon, Aug 13, 2018 at 11:56:26PM +0200, Leon Timmermans wrote:

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

From a quick perusal of local::lib's documentation, it appears that it doesn't use version-specific paths under the local directory, which on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least it doesn't for the common scenario where one has multiple (incompatible) perls and at login time one doesn't know yet which to use when. The only primitive we provide (PERL5LIB) does "prepend these dirs to @INC". I can sort of imagine a tool that does what you suggest, but the additional complexity and fragility sound uncomfortable.

It's a real problem, but I don't think it can be solved on the local::lib side alone.

Thanks for the discussion. I don't have two different versions of Perl installed, just one version from my distribution Arch, which gets updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind what it looks like for a new user who is doing the standard (?) thing of updating Perl via his distribution packaging system, while having CPAN modules installed locally (e.g. in his home directory, is this not typical?).

There seems to be no discussion of changing the mismatched binary error message to point to documentation about e.g. updating CPAN modules.

If the default setup were fixed so that version-specific paths were used for local modules, then what would our user experience consist of - user installs CPAN modules to make his scripts work, then a year later he upgrades his distribution to a new version of Perl and his scripts stop working with "Can't locate XXX.pm in @INC"? I guess that is better than the error message I saw, because then the user at least has a module name to start with, but I think it would be preferable to have an error that somehow ultimately leads (e.g. via a reference to a man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error:

Zlib.c: loadable library and perl binaries are mismatched (got handshake key 0xde00080, needed 0xce00080)

to say:

In Perl module Compress::Raw::Zlib (Zlib.c): loadable library and perl binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do you need to recompile this module for a new Perl version? See `man perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick

Aug 15 '18 00:08 p5pRT

From @grinnz

On Tue, Aug 14, 2018 at 8:16 PM <frederik@ofb.net> wrote:

Thanks for the discussion. I don't have two different versions of Perl installed, just one version from my distribution Arch, which gets updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind what it looks like for a new user who is doing the standard (?) thing of updating Perl via his distribution packaging system, while having CPAN modules installed locally (e.g. in his home directory, is this not typical?).

There seems to be no discussion of changing the mismatched binary error message to point to documentation about e.g. updating CPAN modules.

If the default setup were fixed so that version-specific paths were used for local modules, then what would our user experience consist of - user installs CPAN modules to make his scripts work, then a year later he upgrades his distribution to a new version of Perl and his scripts stop working with "Can't locate XXX.pm in @INC"? I guess that is better than the error message I saw, because then the user at least has a module name to start with, but I think it would be preferable to have an error that somehow ultimately leads (e.g. via a reference to a man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error:
Zlib\.c&#8203;: loadable library and perl binaries are mismatched $got
handshake key 0xde00080, needed 0xce00080)

to say:
In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c$&#8203;: loadable library and perl
binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do you need to recompile this module for a new Perl version? See `man perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick

These are good ideas. The fundamental issue though is that modules you installed for one major version of Perl must always be installed again for another, and users don't tend to expect this whether they use local::lib or not, it will always be a problem if you install modules without using your package manager in a Perl managed by your package manager.

-Dan

Aug 15 '18 00:08 p5pRT

From @eserte

Dana Tue, 14 Aug 2018 16:19:54 -0700, LeonT reče:

On Tue, Aug 14, 2018 at 10:41 AM, Dave Mitchell <davem@iabyn.com> wrote:

On Mon, Aug 13, 2018 at 11:56:26PM +0200, Leon Timmermans wrote:

I think most people avoid configurations where incompatible perl builds share the same arch directories. Judging by your @INC my first guess is that you're reusing your old perl's local::lib directories, which would indeed blow up in your face.

From a quick perusal of local::lib's documentation, it appears that it doesn't use version-specific paths under the local directory, which on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least it doesn't for the common scenario where one has multiple (incompatible) perls and at login time one doesn't know yet which to use when. The only primitive we provide (PERL5LIB) does "prepend these dirs to @INC". I can sort of imagine a tool that does what you suggest, but the additional complexity and fragility sound uncomfortable.

The PERL5LIB mechanism is sufficient here. If set user sets something like

PERL5LIB=/root/perl5/lib/perl5

then @INC contains automatically

/root/perl5/lib/perl5/5.28.0/x86_64-linux-thread-multi /root/perl5/lib/perl5/5.28.0 /root/perl5/lib/perl5/x86_64-linux-thread-multi /root/perl5/lib/perl5

so architecture and perl api (version) specific files have their place.

The problem is, as I said, that INSTALL_BASE does not use these version-specific files.

Aug 15 '18 05:08 p5pRT

From [email protected]

On Tue, Aug 14, 2018 at 05:28:03PM -0700, Dan Book via RT wrote:

On Tue, Aug 14, 2018 at 8:16 PM <frederik@ofb.net> wrote:
Thanks for the discussion. I don't have two different versions of Perl installed, just one version from my distribution Arch, which gets updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind what it looks like for a new user who is doing the standard (?) thing of updating Perl via his distribution packaging system, while having CPAN modules installed locally (e.g. in his home directory, is this not typical?).

There seems to be no discussion of changing the mismatched binary error message to point to documentation about e.g. updating CPAN modules.

If the default setup were fixed so that version-specific paths were used for local modules, then what would our user experience consist of - user installs CPAN modules to make his scripts work, then a year later he upgrades his distribution to a new version of Perl and his scripts stop working with "Can't locate XXX.pm in @INC"? I guess that is better than the error message I saw, because then the user at least has a module name to start with, but I think it would be preferable to have an error that somehow ultimately leads (e.g. via a reference to a man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error:
Zlib\.c&#8203;: loadable library and perl binaries are mismatched $got
handshake key 0xde00080, needed 0xce00080)

to say:
In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c$&#8203;: loadable library and perl
binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do you need to recompile this module for a new Perl version? See `man perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick
These are good ideas. The fundamental issue though is that modules you installed for one major version of Perl must always be installed again for another, and users don't tend to expect this whether they use local::lib or not, it will always be a problem if you install modules without using your package manager in a Perl managed by your package manager.

Thanks. So people who use cpanm tend to have Perl compiled locally? I'm still trying to figure out the common use case.

I didn't realize that you can do "cpanm Perl" but it looks like it fails:

Perl.xs: At top level: Perl.xs:33:27: error: ‘PL_tokenbuf’ undeclared here (not in a function); did you mean ‘PL_top_env’? char ptokenbuf[sizeof(PL_tokenbuf)]; ^~~~~~~~~~~ PL_top_env
I wonder how much space this ends up taking up were it to succeed. I think the idea with having modules installed in my home directory is that other scripts in my home directory can then depend on them, and I can sync the scripts between different hosts along with my shell config and so on.

But that would argue for version-specific paths (because the different hosts might have different Perl versions).

Thank you,

Frederick

Aug 15 '18 05:08 p5pRT

From @andk

On Tue, 14 Aug 2018 22:58:52 -0700, frederik@ofb.net said:

> But that would argue for version-specific paths (because the different > hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after some library upgrade. The shell that comes with CPAN.pm has a commend for that, it is called `recompile`. Maye you try that out. Disclaimer: I wrote it. Let me know if you have questions about it.

-- andreas

Aug 15 '18 06:08 p5pRT

From [email protected]

On Tue, Aug 14, 2018 at 11:47:54PM -0700, (Andreas J. Koenig) via RT wrote:

On Tue, 14 Aug 2018 22:58:52 -0700, frederik@ofb.net said:

But that would argue for version-specific paths (because the different hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after some library upgrade. The shell that comes with CPAN.pm has a commend for that, it is called `recompile`. Maye you try that out. Disclaimer: I wrote it. Let me know if you have questions about it.

That's good advice, although I would say that the main problem for me is users being faced with an error message that doesn't point to a clear solution. But maybe that's what Stack Exchange or IRC is for? And maybe I'm the only one that's confused? Nevertheless one would hope that the local documentation would be sufficient for something this basic.

FWIW I tried your 'recompile' and I got a bunch of errors. I'm attaching the output. I'm rerunning it after "cpanm -U Digest::SHA" and it seems to be doing something productive.

The approach I tried earlier, which I think I mentioned, was to run cpan-outdated, but I guess this is something different - its man page doesn't say anything about helping you recompile binary modules which may not have actually changed versions. Your suggestion seems more promising.

Thank you,

Frederick

Aug 15 '18 07:08 p5pRT

From [email protected]

cpan.out

Aug 15 '18 07:08 p5pRT

From @iabyn

On Tue, Aug 14, 2018 at 05:14:41PM -0700, frederik@ofb.net wrote:

Thanks for the discussion. I don't have two different versions of Perl installed, just one version from my distribution Arch, which gets updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind what it looks like for a new user who is doing the standard (?) thing of updating Perl via his distribution packaging system, while having CPAN modules installed locally (e.g. in his home directory, is this not typical?).

The more typical approach is to have a personal perl installation separate from the system's one: this would contain both the perl binary plus any additional modules you have installed. That way you have complete control over what gets upgraded and when. Using the system perl in combination with privately built/installed modules puts you at the mercy of whatever your OS's update/upgrade procedures do to the system perl.

If the default setup were fixed so that version-specific paths were used for local modules,

Note that the default behaviour for perl *is* always to install in version-specific directories. The behaviour you are seeing is that of a third-party module, local::lib, which is not bundled with perl, and which we (p5p) have no control over. (Although elsewhere in this thread it appears that MakeMaker etc may be to blame too.)

then what would our user experience consist of - user installs CPAN modules to make his scripts work, then a year later he upgrades his distribution to a new version of Perl and his scripts stop working with "Can't locate XXX.pm in @INC"? I guess that is better than the error message I saw, because then the user at least has a module name to start with, but I think it would be preferable to have an error that somehow ultimately leads (e.g. via a reference to a man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error:
Zlib\.c&#8203;: loadable library and perl binaries are mismatched $got handshake key
0xde00080, needed 0xce00080)

to say:
In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c$&#8203;: loadable library and perl binaries are mismatched $got handshake key 0xde00080\, needed 0xce00080$\. Do you need to recompile this module for a new Perl version? See \`man perlmodupgrade\`\.
(Actually, I'm noticing that the "perlmodinstall" manual page doesn't mention cpanm or cpan tools, I wonder if those should be there too)

Some observations.

As I pointed out earlier, I'm not sure that it will be possible (or at least easy) to display the module name as part of the error message.

I'm very much in favour of removing the 'handshake' part of the message, and replacing it with something informative and useful - i.e. to actually decode the hex values and explain what was mismatched.

I'm also not opposed to the error message referring to a man page; although note that if your perl installation is borked due to version mismatches, then it's possible that the tool you use for viewing man pages (such as perldoc) also wont work!

It may be overkill to have a complete new man page - perhaps just an extra section in an existing document instead?

It will be quite difficult to write such a perlmodupgrade man page. Bear in mind that the error message you got refers to any sort of mismatch between the perl binary you are executing, and an XS module which that perl is trying to load. Your particular scenario is just one of many, and reinstalling the offending module isn't always the correct solution. Such a manual page would have to explain many possible scenarios, including interactions with third-party solutions such as OS packages, perlbrew, local::lib etc.

Personally I don't feel I have the expertise to write all of such a document.

-- Any [programming] language that doesn't occasionally surprise the novice will pay for it by continually surprising the expert. -- Larry Wall

Aug 15 '18 10:08 p5pRT

From @grinnz

On Wed, Aug 15, 2018 at 6:36 AM Dave Mitchell <davem@iabyn.com> wrote:

The more typical approach is to have a personal perl installation separate from the system's one: this would contain both the perl binary plus any additional modules you have installed. That way you have complete control over what gets upgraded and when. Using the system perl in combination with privately built/installed modules puts you at the mercy of whatever your OS's update/upgrade procedures do to the system perl.

To add to this: typical methods of doing this are Perl::Build, perlbrew, and plenv. The first simply installs a Perl somewhere, the other two are Perl-based and shell-based multiple-Perl managers.

-Dan

Aug 15 '18 16:08 p5pRT

From [email protected]

Just wanted to briefly follow this up...

Andreas: "cpan[1]> recompile" worked fine on both my machines. I also noticed that the second time I run it, it notices nothing needs to be done, which is great.

Dave: Thanks for the observations. I'm happy to look at the code and send a patch for a better error message when I have time, which is probably not going to be within the next month or so. Also I think it should be worthwhile to investigate a way to get the full module name in the message, presumably whatever compiles modules (MakeMaker?) can be changed to provide a CPP macro with the module name, and I think this could be done in a backwards-compatible manner - whatever that means.

Dan: I can look at perlbrew et al (it was also recommended off-list), I'd of course like to see how far I can go with the binaries provided by my distro.

One problem with my argument that the error message isn't user-friendly, is that anyone can easily submit a bug for it and get friendly and helpful replies from the developers, as I have done. Still, I'm interested in improving things and I hope that I myself or someone else can find time to work on this in the not-too-distant future.

Thanks,

Frederick

On Tue, Aug 14, 2018 at 11:47:54PM -0700, (Andreas J. Koenig) via RT wrote:

On Tue, 14 Aug 2018 22:58:52 -0700, frederik@ofb.net said:

But that would argue for version-specific paths (because the different hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after some library upgrade. The shell that comes with CPAN.pm has a commend for that, it is called `recompile`. Maye you try that out. Disclaimer: I wrote it. Let me know if you have questions about it.

-- andreas

On Wed, Aug 15, 2018 at 03:35:57AM -0700, Dave Mitchell via RT wrote:

On Tue, Aug 14, 2018 at 05:14:41PM -0700, frederik@ofb.net wrote:

Thanks for the discussion. I don't have two different versions of Perl installed, just one version from my distribution Arch, which gets updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind what it looks like for a new user who is doing the standard (?) thing of updating Perl via his distribution packaging system, while having CPAN modules installed locally (e.g. in his home directory, is this not typical?).

The more typical approach is to have a personal perl installation separate from the system's one: this would contain both the perl binary plus any additional modules you have installed. That way you have complete control over what gets upgraded and when. Using the system perl in combination with privately built/installed modules puts you at the mercy of whatever your OS's update/upgrade procedures do to the system perl.

If the default setup were fixed so that version-specific paths were used for local modules,

Note that the default behaviour for perl *is* always to install in version-specific directories. The behaviour you are seeing is that of a third-party module, local::lib, which is not bundled with perl, and which we (p5p) have no control over. (Although elsewhere in this thread it appears that MakeMaker etc may be to blame too.)
then what would our user experience consist of - user installs CPAN modules to make his scripts work, then a year later he upgrades his distribution to a new version of Perl and his scripts stop working with "Can't locate XXX.pm in @INC"? I guess that is better than the error message I saw, because then the user at least has a module name to start with, but I think it would be preferable to have an error that somehow ultimately leads (e.g. via a reference to a man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error:
Zlib\.c&#8203;: loadable library and perl binaries are mismatched $got handshake key
0xde00080, needed 0xce00080)

to say:
In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c$&#8203;: loadable library and perl binaries are mismatched $got handshake key 0xde00080\, needed 0xce00080$\. Do you need to recompile this module for a new Perl version? See \`man perlmodupgrade\`\.
(Actually, I'm noticing that the "perlmodinstall" manual page doesn't mention cpanm or cpan tools, I wonder if those should be there too)
Some observations.

As I pointed out earlier, I'm not sure that it will be possible (or at least easy) to display the module name as part of the error message.

I'm very much in favour of removing the 'handshake' part of the message, and replacing it with something informative and useful - i.e. to actually decode the hex values and explain what was mismatched.

I'm also not opposed to the error message referring to a man page; although note that if your perl installation is borked due to version mismatches, then it's possible that the tool you use for viewing man pages (such as perldoc) also wont work!

It may be overkill to have a complete new man page - perhaps just an extra section in an existing document instead?

It will be quite difficult to write such a perlmodupgrade man page. Bear in mind that the error message you got refers to any sort of mismatch between the perl binary you are executing, and an XS module which that perl is trying to load. Your particular scenario is just one of many, and reinstalling the offending module isn't always the correct solution. Such a manual page would have to explain many possible scenarios, including interactions with third-party solutions such as OS packages, perlbrew, local::lib etc.

Personally I don't feel I have the expertise to write all of such a document.

-- Any [programming] language that doesn't occasionally surprise the novice will pay for it by continually surprising the expert. -- Larry Wall

Aug 17 '18 04:08 p5pRT

From @andk

On Thu, 16 Aug 2018 21:51:39 -0700, frederik@ofb.net said:

> Just wanted to briefly follow this up... > Andreas: "cpan[1]> recompile" worked fine on both my machines. I also > noticed that the second time I run it, it notices nothing needs to be > done, which is great.

Thanks for letting me know and for sending the report how noisy it was during its performance. I'm sorry for this bug, I've since fixed it in the repository so it will appear in version 2.21.

Besides that I have also found the original ticket in which we received the first guide that explained the "got handshake key" message. It was https://rt.perl.org/Public/Bug/Display.html?id=125236. Maybe the two tickets would be suited to be folded into one, I'm not sure.

-- andreas

Aug 17 '18 20:08 p5pRT

From @karenetheridge

On Wed, Aug 15, 2018 at 3:35 AM, Dave Mitchell <davem@iabyn.com> wrote:

Note that the default behaviour for perl *is* always to install in version-specific directories. The behaviour you are seeing is that of a third-party module, local::lib, which is not bundled with perl, and which we (p5p) have no control over. (Although elsewhere in this thread it appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group and p5p. We should not go "oh well, not in core, not our problem" and leave it to the original bug reporter to have to pursue the issue in another bug queue (and attempt to explain the problem again, which leads to a game of telephone). We can do better than that.

All local::lib does is set some environment variables, e.g.:

$ cd ~; perl -Mlocal::lib=local Attempting to create directory /Users/ether/local PATH="/Users/ether/local/bin${PATH:+:${PATH}}"; export PATH; PERL5LIB="/Users/ether/local/lib/perl5${PERL5LIB:+:${PERL5LIB}}"; export PERL5LIB; PERL_LOCAL_LIB_ROOT="/Users/ether/local${PERL_LOCAL_LIB_ROOT:+:${PERL_LOCAL_LIB_ROOT}}"; export PERL_LOCAL_LIB_ROOT; PERL_MB_OPT="--install_base \"/Users/ether/local\""; export PERL_MB_OPT; PERL_MM_OPT="INSTALL_BASE=/Users/ether/local"; export PERL_MM_OPT; Perhaps those environment variables should always contain the perl version. There would be some backcompat issues, but we could possibly handle that by putting the old non-versioned directory into PERL5LIB after the new versioned one (or something clever with symlinks, or.. well, we'd have to discuss it.)

-ether@cpan.org

Aug 19 '18 02:08 p5pRT

From [email protected]

Thanks for your reply. While trying to downgrade my system (debugging another issue) I discovered a possible reason why I have problematic binary modules in my home directory which are also in Perl core (causing cpan to break when I upgrade Perl). The reason appears to be that the "recompile" cpan command reinstalls modules which I had removed earlier using 'cpanm'.

$ PERL5LIB= cpanm -U Digest::SHA ... $ PERL5LIB= cpanm -U Time::HiRes ... $ perldoc -l Time::HiRes /usr/lib/perl5/5.28/core_perl/Time/HiRes.pm $ perldoc -l Digest::SHA /usr/lib/perl5/5.28/core_perl/Digest/SHA.pm

(after cpan recompile)

$ perldoc -l Time::HiRes /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Time/HiRes.pm $ perldoc -l Digest::SHA /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Digest/SHA.pm

I don't know if this is a bug or just an annoyance, but it may go a little way towards solving the mystery of why I have these modules around, since I think I recall removing them earlier. Obviously, I don't look too carefully at the ouput of 'cpan -r', because I have better things to do, I just assume it's doing the "right thing". By the time I realize that I have duplicate modules installed, it's probably a year later and I don't remember having removed them.

Also, I notice that if I ask 'cpanm' to install a module which is already in core, then it doesn't warn me:

$ perldoc -l DB_File /usr/lib/perl5/5.28/core_perl/DB_File.pm $ cpanm DB_File --> Working on DB_File Fetching http://www.cpan.org/authors/id/P/PM/PMQS/DB_File-1.842.tar.gz ... OK Configuring DB_File-1.842 ... ^C

I haven't checked what happens if I use cpanm to install a module which depends on modules in core, but if you give me an example of a CPAN module with dependencies in core then I can try this out.

I couldn't figure out how to use the 'cpan' tool to remove modules, but 'cpanm' has a nice simple interface for doing that so I thought it would be compatible with 'cpan'.

Also, I see 'cpan -r -T' runs lots of tests, I thought '-T' was to supposed to skip the tests.

Hope this helps,

Frederick

On Fri, Aug 17, 2018 at 01:40:52PM -0700, (Andreas J. Koenig) via RT wrote:

On Thu, 16 Aug 2018 21:51:39 -0700, frederik@ofb.net said:

Just wanted to briefly follow this up... Andreas: "cpan[1]> recompile" worked fine on both my machines. I also noticed that the second time I run it, it notices nothing needs to be done, which is great.

Thanks for letting me know and for sending the report how noisy it was during its performance. I'm sorry for this bug, I've since fixed it in the repository so it will appear in version 2.21.

Besides that I have also found the original ticket in which we received the first guide that explained the "got handshake key" message. It was https://rt.perl.org/Public/Bug/Display.html?id=125236. Maybe the two tickets would be suited to be folded into one, I'm not sure.

Aug 19 '18 03:08 p5pRT

From @grinnz

On Sat, Aug 18, 2018 at 11:26 PM <frederik@ofb.net> wrote:

Thanks for your reply. While trying to downgrade my system (debugging another issue) I discovered a possible reason why I have problematic binary modules in my home directory which are also in Perl core (causing cpan to break when I upgrade Perl). The reason appears to be that the "recompile" cpan command reinstalls modules which I had removed earlier using 'cpanm'.

$ PERL5LIB= cpanm -U Digest::SHA ... $ PERL5LIB= cpanm -U Time::HiRes ... $ perldoc -l Time::HiRes /usr/lib/perl5/5.28/core_perl/Time/HiRes.pm $ perldoc -l Digest::SHA /usr/lib/perl5/5.28/core_perl/Digest/SHA.pm

(after cpan recompile)

$ perldoc -l Time::HiRes /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Time/HiRes.pm $ perldoc -l Digest::SHA /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Digest/SHA.pm

I don't know if this is a bug or just an annoyance, but it may go a little way towards solving the mystery of why I have these modules around, since I think I recall removing them earlier. Obviously, I don't look too carefully at the ouput of 'cpan -r', because I have better things to do, I just assume it's doing the "right thing". By the time I realize that I have duplicate modules installed, it's probably a year later and I don't remember having removed them.

Also, I notice that if I ask 'cpanm' to install a module which is already in core, then it doesn't warn me:

$ perldoc -l DB_File /usr/lib/perl5/5.28/core_perl/DB_File.pm $ cpanm DB_File --> Working on DB_File Fetching http://www.cpan.org/authors/id/P/PM/PMQS/DB_File-1.842.tar.gz ... OK Configuring DB_File-1.842 ... ^C

I haven't checked what happens if I use cpanm to install a module which depends on modules in core, but if you give me an example of a CPAN module with dependencies in core then I can try this out.

I couldn't figure out how to use the 'cpan' tool to remove modules, but 'cpanm' has a nice simple interface for doing that so I thought it would be compatible with 'cpan'.

These are dual-life modules; it is expected and often desired to install them a second time in site or vendor libs to upgrade them from the core module version. Since Perl 5.12, site and vendor libs take precedence in @INC for that reason. local::lib of course takes precedence over all standard lib directories. cpanm's uninstall command relies on packlists, and thus can only uninstall modules installed by a cpan client (whether cpanm or cpan).

-Dan

Aug 19 '18 06:08 p5pRT

From [email protected]

These are dual-life modules; it is expected and often desired to install them a second time in site or vendor libs to upgrade them from the core module version. Since Perl 5.12, site and vendor libs take precedence in @INC for that reason. local::lib of course takes precedence over all standard lib directories. cpanm's uninstall command relies on packlists, and thus can only uninstall modules installed by a cpan client (whether cpanm or cpan).

You're saying that these modules are purposefully installed by 'cpan' even when compatible versions already exist in 'core'?

I guess not everyone expects this, because when I first submitted this bug report I got a response from Andy Dougherty saying "I'm not sure why you have Compress::Raw::Zlib installed locally, since it has been bundled with perl since version 5.9.4." And I have in my shell history that I uninstalled it, while now it is back in ~/.local. Although when I removed it again and re-ran 'cpan -r', it didn't reappear, so I'm not entirely sure what's happening.

Thanks,

Frederick

Aug 21 '18 23:08 p5pRT

From @grinnz

On Tue, Aug 21, 2018 at 7:45 PM <frederik@ofb.net> wrote:

You're saying that these modules are purposefully installed by 'cpan' even when compatible versions already exist in 'core'?

I guess not everyone expects this, because when I first submitted this bug report I got a response from Andy Dougherty saying "I'm not sure why you have Compress::Raw::Zlib installed locally, since it has been bundled with perl since version 5.9.4." And I have in my shell history that I uninstalled it, while now it is back in ~/.local. Although when I removed it again and re-ran 'cpan -r', it didn't reappear, so I'm not entirely sure what's happening.

That is correct. His response seemed to account for this possibility but was asking why you had installed a newer version in a local lib without the use of version specific directories, which could be considered a deficiency in the toolchain. Regardless, upgrades to dual-life modules are installed in sitelib or local libs very commonly, this is the reason they are available from CPAN. I can't comment on how CPAN.pm's recompile function interacts with dual-life modules or local libs.

-Dan

Aug 22 '18 00:08 p5pRT

From [email protected]

By the way I never took the time to thank you for pointing these things out and trying to up the responsibility level a bit here.

Thanks!

Frederick

On Sat, Aug 18, 2018 at 07:55:29PM -0700, Karen Etheridge via RT wrote:

On Wed, Aug 15, 2018 at 3:35 AM, Dave Mitchell <davem@iabyn.com> wrote:

Note that the default behaviour for perl *is* always to install in version-specific directories. The behaviour you are seeing is that of a third-party module, local::lib, which is not bundled with perl, and which we (p5p) have no control over. (Although elsewhere in this thread it appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group and p5p. We should not go "oh well, not in core, not our problem" and leave it to the original bug reporter to have to pursue the issue in another bug queue (and attempt to explain the problem again, which leads to a game of telephone). We can do better than that.

All local::lib does is set some environment variables, e.g.:

$ cd ~; perl -Mlocal::lib=local Attempting to create directory /Users/ether/local PATH="/Users/ether/local/bin${PATH:+:${PATH}}"; export PATH; PERL5LIB="/Users/ether/local/lib/perl5${PERL5LIB:+:${PERL5LIB}}"; export PERL5LIB; PERL_LOCAL_LIB_ROOT="/Users/ether/local${PERL_LOCAL_LIB_ROOT:+:${PERL_LOCAL_LIB_ROOT}}"; export PERL_LOCAL_LIB_ROOT; PERL_MB_OPT="--install_base \"/Users/ether/local\""; export PERL_MB_OPT; PERL_MM_OPT="INSTALL_BASE=/Users/ether/local"; export PERL_MM_OPT; Perhaps those environment variables should always contain the perl version. There would be some backcompat issues, but we could possibly handle that by putting the old non-versioned directory into PERL5LIB after the new versioned one (or something clever with symlinks, or.. well, we'd have to discuss it.)

-ether@cpan.org

Aug 22 '18 00:08 p5pRT

From @iabyn

On Sat, Aug 18, 2018 at 07:55:14PM -0700, Karen Etheridge wrote:

On Wed, Aug 15, 2018 at 3:35 AM, Dave Mitchell <davem@iabyn.com> wrote:

Note that the default behaviour for perl *is* always to install in version-specific directories. The behaviour you are seeing is that of a third-party module, local::lib, which is not bundled with perl, and which we (p5p) have no control over. (Although elsewhere in this thread it appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group and p5p. We should not go "oh well, not in core, not our problem" and leave it to the original bug reporter to have to pursue the issue in another bug queue (and attempt to explain the problem again, which leads to a game of telephone). We can do better than that.

You quoted one paragraph of mine from a long email, where I had clearly spent some some and thought on the issue, and was attempting to narrow the issues, and trying to find out which bits need fixing and by who etc. At no point did I say or imply "not a perl issue, go bug the owner of local::lib instead; I'm closing this ticket".

What I said in that paragraph is also completely true - whatever discussions we might have here, and what grand solutions we might propose to fix local::lib, ultimately the owner of the cpan module is free to have their own ideas as to what what their module should do and be irritated that p5p is drying to dictate solutions to them.

So I'm mildly peeved by your response, and the OP's follow up of "oh at last someone's taking some responsibility" - after about 7 porters have already taken time and effort to try and diagnose the issue or suggest workarounds and/or long-term solutions.

-- "There's something wrong with our bloody ships today, Chatfield." -- Admiral Beatty at the Battle of Jutland, 31st May 1916.

Aug 22 '18 08:08 p5pRT

From [email protected]

So I'm mildly peeved by your response, and the OP's follow up of "oh at last someone's taking some responsibility" - after about 7 porters have already taken time and effort to try and diagnose the issue or suggest workarounds and/or long-term solutions.

I'm sorry Dave, my reply to Karen was meant to be off-list, but I failed to note the "via RT" in the email address. (Apologies to Karen as well)

In any case I wasn't thanking her for taking responsibility, but for encouraging P5P to do so. I'm also very familiar with the annoyance of having to "pursue the issue in another bug queue", although I'm not sure where I would send what suggestions, as I think we haven't really discussed that yet.

I'm attaching some proposed patches for P5P, maybe I have enough information to take some responsibility as well at this point. However, these are not meant to be a "finished product", I'm sure they will need some additional work before they can be applied.

I would hope that eventually the full module name can be reported by the util.c error message, but rather than dig around to figure out how to do that (I am badly unfamiliar with the software) I decided that my documentation patch would just explain how to search for the module using the information supplied by the current message.

Thank you,

Frederick

Aug 22 '18 16:08 p5pRT

From [email protected]

util.c.diff

--- perl-5.28.0/util.c	2018-05-21 05:29:23.000000000 -0700
+++ /home/frederik/util.c-mod	2018-08-22 09:19:26.797143798 -0700
@@ -5337,7 +5337,8 @@
 	bad_handshake:/* recycle branch and string from above */
 	if(got != (void *)HSf_NOCHK)
 	    noperl_die("%s: loadable library and perl binaries are mismatched"
-                       " (got handshake key %p, needed %p)\n",
+                       " (got handshake key %p, needed %p)."
+                       " See perldiag(1) (do you need to recompile?)\n",
 		file, got, need);
     }

Aug 22 '18 16:08 p5pRT

From [email protected]

perldiag.diff

--- perl-5.28.0/pod/perldiag.pod	2018-05-21 05:29:23.000000000 -0700
+++ /home/frederik/perldiag-mod	2018-08-22 09:34:08.050699448 -0700
@@ -3365,11 +3365,52 @@
 
 =item %s: loadable library and perl binaries are mismatched (got handshake key %p, needed %p)
 
-(P) A dynamic loading library C<.so> or C<.dll> was being loaded into the
-process that was built against a different build of perl than the
-said library was compiled against.  Reinstalling the XS module will
+(P) A dynamic loading library C<.so> or C<.dll> was being loaded into
+the process that was built against a different build of perl than the
+said library was compiled against. Reinstalling the XS module will
 likely fix this error.
 
+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet. In this case the problem can be fixed by telling the
+cpan tool to recompile all local modules. For example, on Unix-based
+installations:
+
+    $ PERL5LIB= cpan -r
+
+(the "PERL5LIB=" prefix ensures that cpan doesn't try to use local
+modules which are assumed to be broken in this case). This command
+should be run after every distribution upgrade.
+
+Alternatively, heavy users of modules downloaded from CPAN can install
+a local Perl instance using such tools as Perl::Build, perlbrew, and
+plenv, and use these when running local scripts. This will allow
+locally-installed Perl software to keep working when the system-wide
+Perl is upgraded, by managing two (or more) distinct versions of the
+Perl interpreter in parallel.
+
+To deal with this error in a module-specific way, the location of the
+offending module can be found by searching in your PERL5LIB for a file
+with the same name as is listed in the error message, for example:
+
+    Parser.c: loadable library and perl binaries are mismatched (got handshake key 0xce00080, needed 0xde00080)
+    # Find the DLL, looks like it belongs to the HTML::Parser module
+    $ find .local/lib/perl5 | grep Parser.so
+    .local/lib/perl5/x86_64-linux-thread-multi/auto/HTML/Parser/Parser.so
+    # Check that this is really where the module is
+    $ perldoc -l HTML::Parser
+    ~/.local/lib/perl5/x86_64-linux-thread-multi/HTML/Parser.pm
+    # Notice that the module is also installed system-wide
+    $ PERL5LIB= perldoc -l HTML::Parser
+    /usr/lib/perl5/5.26/vendor_perl/HTML/Parser.pm
+    # Remove the local version of the library (if installed via cpan)
+    $ PERL5LIB= cpanm -U HTML::Parser
+    ...
+    # Download, compile and install the newest version of the library (if
+    # needed)
+    $ PERL5LIB= cpanm HTML::Parser
+
 =item Locale '%s' contains (at least) the following characters which
 have unexpected meanings: %s  The Perl program will use the expected
 meanings

Aug 22 '18 16:08 p5pRT

perl5
perl5 copied to clipboard

binaries mismatched again

From [email protected]

Created by [email protected]

From @doughera88

From [email protected]

From @bulk88

From @Leont

From @iabyn

From @eserte

From @Leont

From [email protected]

From @grinnz

From @eserte

From [email protected]

From @andk

From [email protected]

From [email protected]

From @iabyn

From @grinnz

From [email protected]

From @andk

From @karenetheridge

From [email protected]

From @grinnz

From [email protected]

From @grinnz

From [email protected]

From @iabyn

From [email protected]

From [email protected]

From [email protected]

perl5 perl5 copied to clipboard

binaries mismatched again

From [email protected]

Created by [email protected]

From @doughera88

From [email protected]

From @bulk88

From @Leont

From @iabyn

From @eserte

From @Leont

From [email protected]

From @grinnz

From @eserte

From [email protected]

From @andk

From [email protected]

From [email protected]

From @iabyn

From @grinnz

From [email protected]

From @andk

From @karenetheridge

From [email protected]

From @grinnz

From [email protected]

From @grinnz

From [email protected]

From @iabyn

From [email protected]

From [email protected]

From [email protected]

perl5
perl5 copied to clipboard