perl5
perl5 copied to clipboard
Perl
build fails when building with PERL_NO_COW
Noticed when attempting to build with -Accflags='-DPERL_NO_COW'
$ ./Configure -des -Dusedevel -Accflags='-DPERL_NO_COW'
$ make -j8 miniperl
...
./miniperl -Ilib -f write_buildcustomize.pl
makefile:363: recipe for target 'lib/buildcustomize.pl' failed
make: *** [lib/buildcustomize.pl] Segmentation fault (core dumped)
Rebuilding with -DDEBUGGING shows an assertion failure:
$ ./Configure -des -Dusedevel -Accflags='-DPERL_NO_COW' -DDEBUGGING
$ make -j miniperl
...
./miniperl -Ilib -f write_buildcustomize.pl
miniperl: hv.c:2972: S_unshare_hek_or_pvn: Assertion `he->shared_he_he.hent_hek == hek' failed.
makefile:363: recipe for target 'lib/buildcustomize.pl' failed
Reducing to a minimal test case:
$ ./miniperl -wle 'my $x = (0 == 1);'
miniperl: hv.c:2972: S_unshare_hek_or_pvn: Assertion `he->shared_he_he.hent_hek == hek' failed.
Aborted (core dumped)
Bisecting points to commit 914bb57489325d34ddbb7c0557c53df7baa84d86: (@leonerd)
commit 914bb57489325d34ddbb7c0557c53df7baa84d86
Author: Paul "LeoNerd" Evans <[email protected]>
Date: Sat Aug 7 14:46:48 2021 +0100
Define a third kind of COW state; STATIC
Previously, when IsCOW flag was set there were two cases:
SvLEN()==0:
PV is really a shared HEK
SvLEN()!=0:
PV is a COW structure with 1..256 refcount stored in its extra final byte
This change adds a third state:
SvLEN()==0 && SvFLAGS() & SVppv_STATIC:
PV is a shared static const pointer and must not be modified
sv_setsv_flags() and sv_setsv_cow() will preserve this state
sv_uncow() will copy it out to a regular string buffer
sv_dup() will preserve the static pointer into cloned threads
(Note: I have no use case for -DPERL_NO_COW, I only tried it while looking at another issue)
Steps to reproduce
./Configure -des -Dusedevel -Accflags='-DPERL_NO_COW' -DDEBUGGINGmake -j8 miniperl./miniperl -wle 'my $x = (0 == 1);'
Actual result
Segmentation fault/Assertion failure:
S_unshare_hek_or_pvn: Assertion `he->shared_he_he.hent_hek == hek' failed.
Expected result
No crash.
Noticed when attempting to build with
-Accflags='-DPERL_NO_COW'$ ./Configure -des -Dusedevel -Accflags='-DPERL_NO_COW' $ make -j8 miniperl ... ./miniperl -Ilib -f write_buildcustomize.pl makefile:363: recipe for target 'lib/buildcustomize.pl' failed make: *** [lib/buildcustomize.pl] Segmentation fault (core dumped)
Observed on FreeBSD as well.
On Sat, Jul 23, 2022 at 01:16:13PM -0700, Bram wrote:
Noticed when attempting to build with
-Accflags='-DPERL_NO_COW'$ ./Configure -des -Dusedevel -Accflags='-DPERL_NO_COW' $ make -j8 miniperl ... ./miniperl -Ilib -f write_buildcustomize.pl makefile:363: recipe for target 'lib/buildcustomize.pl' failed make: *** [lib/buildcustomize.pl] Segmentation fault (core dumped)
Given that it's been broken for a year and you're the first to notice, I wonder whether the time has come to remove the facility to build a non-COW perl? It's been on by default for 8 years now.
-- All wight. I will give you one more chance. This time, I want to hear no Wubens. No Weginalds. No Wudolf the wed-nosed weindeers. -- Life of Brian
It's broken in a dev release since v5.35.4 (2021-Sep-20) In a non-dev release it's been broken since v5.36.0 (2022-05-27)
Anyway, briefly discussing this with @leonerd :
- Boolean need copy on write
- firstclass bool values are cow-static strings
The question is if someone still has a use-case for building without Copy on Write.. (I do not have one) If someone does have then it should show up when they attempt to build perl v5.36.. If nothing shows up then "disabling CoW" could be ripped out..
I have delayed upgrading to newer perl because they fail when PERL_NO_COW is on. But that too is now failing, because older perls have other compilation errors on the latest Linux systems. COW does not seem to extend to C functions: when passing a string reference to a C function that updates it (as a byte array) then the cloning does not happen it seems. If this feature disappears I will have to do explicit copying in many places. I think this compilation problem should be fixed (same as above, except on Linux Mint 21.1) rather than shipping for a long time something that doesn't work. Whether the feature is useful or not, thats a separate discussion. I think it is. So my big software package is stuck between one compilation error or another. I will even pay reasonably to have it fixed. (Btw, the switch to Github instead of email means shifting convenience from users to developers, maybe not a good idea, at least I always try to do the opposite with my code).
@nielsl I suspect you just need to change your code a bit so that SvPV() are changed to be one of the following:
char* SvPV_force (SV* sv, STRLEN len)
char* SvPV_force_nolen (SV* sv)
char* SvPVx_force (SV* sv, STRLEN len)
char* SvPV_force_nomg (SV* sv, STRLEN len)
char* SvPV_force_nomg_nolen (SV * sv)
char* SvPV_force_mutable (SV * sv, STRLEN len)
char* SvPV_force_flags (SV * sv, STRLEN len, U32 flags)
char* SvPV_force_flags_nolen (SV * sv, U32 flags)
char* SvPV_force_flags_mutable(SV * sv, STRLEN len, U32 flags)
char* SvPVbyte_force (SV* sv, STRLEN len)
char* SvPVbytex_force (SV* sv, STRLEN len)
char* SvPVutf8_force (SV* sv, STRLEN len)
char* SvPVutf8x_force (SV* sv, STRLEN len)
You can also use the function sv_force_normal() as well. See perldoc perlapi for details.
COW does not refer to the Unix COW facility, it is purely an internal feature.
I would be happy to discuss more privately, you can find my email on the list.
Having said that, I do think that we should make PERL_NO_COW work. Nothing we do should depend on COW. That would be/is a very sad state of affairs, as it would mean we have a hard dependency on some particular implementation of COW, and IMO we should be able to change how COW works without causing havok. The whole scheme of putting the COW bytes at the end of the PV buffer, and various other aspects of its implementation definitely could be improved, as it leaves cases that can't be COW'ed and causes other problems (there is some kind of hairy interaction with file operations and regexes that we never really fixed afair), and if we have somehow hard coded a dependency on COW then IMO we should fix it.
@bram-perl said:
Anyway, briefly discussing this with @leonerd :
- Boolean need copy on write
- firstclass bool values are cow-static strings
That doesn't make sense to me, it feels like an unforced issue that we can resolve if we choose to. I will add this to my todo list.
That doesn't make sense to me, it feels like an unforced issue that we can resolve if we choose to. I will add this to my todo list.
Yah; firstclass-bools don't actually rely on COW, they just happen to exploit the fact that if they set the COW flag, then the PV buffer is shared by plain pointer address, and so we can use that pointer to distinguish if it is the special boolean true/false value. You could arrange for that pointer to be statically-shared by some other mechanism if COW was not enabled.
@demerphq Thank you for giving me a way out at the C level. I am not very good with Perl/C, but managed to create "pidgin-C" functions in all the places where there is a memory or speed bottleneck, and where the code doesn't change.
@bram-perl said:
Anyway, briefly discussing this with @leonerd :
- Boolean need copy on write
- firstclass bool values are cow-static strings
That doesn't make sense to me, it feels like an unforced issue that we can resolve if we choose to. I will add this to my todo list.
Oh that would be wonderful, thanks. I can then update to the latest Perl and to all the non-standard Perl and GNU packages that I depend on. Then I can install my system on the latest Linux and Unix systems again. And if the fix is not immediate, I can now make temporary workarounds knowing that there will be a fix. Bravo.