perl5
perl5 copied to clipboard
Perl
How do I configure with ASAN?
More than once, I have had to ask, "How do I invoke ./Configure so as to build with ASAN?" (Most recently: https://github.com/Perl/perl5/pull/22586#issuecomment-2344507640) But I never seem to succeed. Can we straighten this out so that more people can explore address sanitizers?
My environment:
$ uname -mrs
Linux 6.8.0-40-generic x86_64
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ git describe
v5.41.3
My most recent attempt (from https://github.com/Perl/perl5/pull/22586#issuecomment-2344716848):
$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=clang
...
./Configure gets almost all the way through, then segfaults:
...
Run make depend now? [y]
clang -c -DPERL_CORE -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall generate_uudmap.c
clang -o generate_uudmap -fsanitize=address -fstack-protector-strong -L/usr/local/lib generate_uudmap.o -lpthread -ldl -lm -lcrypt -lutil -lc
./generate_uudmap uudmap.h bitcount.h mg_data.h
make: *** [Makefile:346: bitcount.h] Segmentation fault (core dumped)
...
I should note in passing that it feels awkward to me to delay calling -Dcc=clang until after I've called -Accflags and -Aldflags. But when I put -Dcc=clang where it feels more natural, ./Configure quickly collapses:
$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address
...
Let me guess what the preprocessor flags are...
Any additional ld flags (NOT including libraries)?
[ -fsanitize=address -fstack-protector-strong -L/usr/local/lib]
Checking your choice of C compiler and flags for coherency...
I've tried to compile and run the following simple program:
#include <stdio.h>
int main() { printf("Ok\n"); return(0); }
I used the command:
clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc
./try
and I got the following output:
Segmentation fault (core dumped)
The program compiled OK, but exited with status 139.
You have a problem. Shall I abort Configure [y]
Ok. Stopping Configure.
Next, let's try the above only with gcc instead of clang:
$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=gcc -Accflags=-fsanitize=address -Aldflags=-fsanitize=address
...
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
...
^C
I get what appears to be an infinite loop of those DEADLYSIGNALs.
And finally, putting gcc at the end of the invocation:
$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=gcc
...
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
...
^C
I have a vague recollection that in the past I was able to complete ./Configure only to have my machine grind to a halt during make. But now I can't even get through ./Configure!
Guidance requested.
On 9/12/24 16:09, James E Keenan wrote:
More than once, I have had to ask, "How do I invoke |./Configure| so as to build with ASAN?" (Most recently: #22586 (comment) https://github.com/Perl/perl5/pull/22586#issuecomment-2344507640) But I never seem to succeed. Can we straighten this out so that more people can explore address sanitizers?
My environment:
|$ uname -mrs Linux 6.8.0-40-generic x86_64 $ clang --version Ubuntu clang version 14.0.0-1ubuntu1.1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin $ gcc --version gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ git describe v5.41.3 |
My most recent attempt (from #22586 (comment) https://github.com/Perl/perl5/pull/22586#issuecomment-2344716848):
|$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=clang ... |
|./Configure| gets almost all the way through, then segfaults: This is my precise invocation that worked just now
bin/bash ./Configure -des -Uversiononly '-Dprefix=/home/khw/perl/devel' -Dusedevel '-Doptimize=-O0' -DDEBUGGING $'-A'optimize=-g3'' $'-A'optimize=-ggdb3'' $'-A'optimize=-g3'' $'-Accflags='-fsanitize=address'' $'-Accflags='-fsanitize-address-use-after-scope'' $'-Accflags='-fno-omit-frame-pointer'' $'-Accflags='-fno-common'' $'-Accflags='-ftrapv'' $'-Accflags='-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore'' $'-Accflags='-Wno-deprecated'' $'-Accflags='-DNO_NL_LOCALE_NAME'' $'-Dman1dir='none'' $'-Dman3dir='none'' $'-Aldflags='-fsanitize=address'' $'-Alddlflags='-fsanitize=address'' $'-Aldflags='-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore'' $'-Alddlflags='-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore'' $'-Alddlflags='-shared'' '-Dcc=clang' -Dusecbacktrace -Dusethreads
Some of these are obsolete or not generally applicable. asan_ignore is shipped with perl.
|... Run make depend now? [y] clang -c -DPERL_CORE -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall generate_uudmap.c clang -o generate_uudmap -fsanitize=address -fstack-protector-strong -L/usr/local/lib generate_uudmap.o -lpthread -ldl -lm -lcrypt -lutil -lc ./generate_uudmap uudmap.h bitcount.h mg_data.h make: *** [Makefile:346: bitcount.h] Segmentation fault (core dumped) ... |
I should note in passing that it feels awkward to me to delay calling |-Dcc=clang| until after I've called |-Accflags| and |-Aldflags|. But when I put |-Dcc=clang| where it feels more natural, |./Configure| quickly collapses:
|$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address ... Let me guess what the preprocessor flags are... Any additional ld flags (NOT including libraries)? [ -fsanitize=address -fstack-protector-strong -L/usr/local/lib] Checking your choice of C compiler and flags for coherency... I've tried to compile and run the following simple program: #include <stdio.h> int main() { printf("Ok\n"); return(0); } I used the command: clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc ./try and I got the following output: Segmentation fault (core dumped) The program compiled OK, but exited with status 139. You have a problem. Shall I abort Configure [y] Ok. Stopping Configure. |
Next, let's try the above only with |gcc| instead of |clang|:
|$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=gcc -Accflags=-fsanitize=address -Aldflags=-fsanitize=address ... AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL ... ^C |
I get what appears to be an infinite loop of those |DEADLYSIGNAL|s.
And finally, putting |gcc| at the end of the invocation:
|$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=gcc ... AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL ... ^C |
I have a vague recollection that in the past I was able to complete |./Configure| only to have my machine grind to a halt during |make|. But now I can't even get through |./Configure|!
Guidance requested.
— Reply to this email directly, view it on GitHub https://github.com/Perl/perl5/issues/22588, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2DH5PT3MK7LDFQTBQP6TZWIGKRAVCNFSM6AAAAABOEFI2BSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUZDGNBUGQ3DQMQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>
I can't reproduce your segfaults. I've just built perl from a clean git directory using these exact commands:
./Configure -Dcc=clang -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des
make
Unsurprisingly, gcc also works here:
./Configure -Dcc=gcc -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des
make
In either case, I get a working perl:
$ ./perl -v
This is perl 5, version 41, subversion 4 (v5.41.4 (v5.41.3-102-gbd12780ddd)) built for x86_64-linux
...
My environment:
$ uname -mrs
Linux 6.8.0-105041-tuxedo x86_64
$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ git describe
v5.41.3-102-gbd12780ddd
On my Ubuntu Linux laptop (where the original complaint) was run, I tried @mauke's invocation and got the same failures as before.
Much to my surprise, I got farther on two servers to which I have access.
Debian 11
$ uname -mrs
Linux 6.1.0-23-amd64 x86_64
$ gcc --version | head -n 1
gcc (Debian 12.2.0-14) 12.2.0
$ clang --version
Debian clang version 14.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ ./Configure -Dcc=gcc -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des && make test_prep
[# Completed successfully ]
$ make test_harness
...
Test Summary Report
-------------------
re/pat.t (Wstat: 9 (Signal: KILL) Tests: 276 Failed: 0)
Non-zero wait status: 9
Parse errors: Bad plan. You planned 1295 tests but ran 276.
re/pat_psycho.t (Wstat: 9 (Signal: KILL) Tests: 1 Failed: 0)
Non-zero wait status: 9
Parse errors: Bad plan. You planned 15 tests but ran 1.
Files=2889, Tests=1184831, 1088 wallclock secs (149.93 usr 40.12 sys + 1170.84 cusr 369.65 csys = 1730.54 CPU)
Result: FAIL
ISTR that at times in the past we've seen failures in t/re/pat.t and t/re/pat_psycho.t under ASAN.
FreeBSD 13
$ uname -mrs
FreeBSD 13.3-RELEASE-p6 amd64
$ gcc --version | head -n 1
gcc (FreeBSD Ports Collection) 13.2.0
$ clang --version
FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367)
Target: x86_64-unknown-freebsd13.3
Thread model: posix
InstalledDir: /usr/bin
$ ./Configure -Dcc=clang -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des && make
...
Everything is up to date. Type 'make test' to run test suite.
$ make test_prep
...
$ make test_harness
...
All tests successful.
Files=2900, Tests=1186258, 823 wallclock secs (264.85 usr 46.89 sys + 2030.75 cusr 373.27 csys = 2715.77 CPU)
Result: PASS
I have a suspicion that this is related to these problems reported to the 'actions' project on GitHub:
https://github.com/actions/runner-images/issues/9491
https://github.com/actions/runner-images/issues/9524
... but the issues discussed there are way outside my area of expertise.
On Mon, Sep 16, 2024 at 01:15:46PM -0700, James E Keenan wrote:
On my Ubuntu Linux laptop (where the original complaint) was run, I tried @mauke's invocation and got the same failures as before.
Much to my surprise, I got farther on two servers to which I have access.
Note that ASAN requires a lot more memory than a normal build. For example on my 16Gb laptop I normally build+test with -j16, but under ASAN I run with -j4 - any more, and I see tests start to fail.
-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.
I've tried to compile and run the following simple program: #include <stdio.h> int main() { printf("Ok\n"); return(0); } I used the command: clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc ./try
What do you see if you try copying the code to a file and compiling/running it yourself? Does it differ if you remove all the -l arguments?
If that's failing it seems like you have a toolchain problem, not a perl problem.
What do you see if you try copying the code to a file and compiling/running it yourself? Does it differ if you remove all the
-larguments?If that's failing it seems like you have a toolchain problem, not a perl problem.
$ uname -mrs
Linux 6.8.0-45-generic x86_64
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ cat gh-22588-try.c
#include <stdio.h>
int main() { printf("Ok\n"); return(0); }
$ clang -o try -O2 -fsanitize=address -fno-strict-aliasing \
-pipe -fstack-protector-strong -I/usr/local/include \
-fsanitize=address -fstack-protector-strong \
-L/usr/local/lib $P5P_DIR/gh-22588-try.c \
-lpthread -ldl -lm -lcrypt -lutil -lc
$ ./try
Ok
$ rm ./try
$ clang -o try -O2 -fsanitize=address -fno-strict-aliasing \
-pipe -fstack-protector-strong -I/usr/local/include \
-fsanitize=address -fstack-protector-strong \
-L/usr/local/lib $P5P_DIR/gh-22588-try.c
$ ./try
Ok
Exactly which OS are you running this on? Ubuntu something, but which version?
Exactly which OS are you running this on? Ubuntu something, but which version?
22.04 LTS
$ uname -a
Linux tpadx1 6.8.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep 11 15:25:05 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
I expect to upgrade to 24.04 within 2 months.
I suspect you have a damaged install, you could try:
sudo dpkg -V
though you'll probably see it complain about many config files.
I upgraded to Ubuntu 24.04 LTS last week, which brought along upgrades to clang and gcc.
$ uname -mrs
Linux 6.8.0-47-generic x86_64
$ lsb_release -a
...
Description: Ubuntu 24.04.1 LTS
...
$ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
...
$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
...
I was able to configure and build (thru make test_prep) successfully with these commands:
$ git describe
v5.41.5-17-g161ff7336a
$ ./perl -Ilib -V:config_args
config_args='-des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address';
As expected, make test_harness experienced failures, likely due to memory, on programs like t/re/pat_psycho.t and slowed the machine to a mini-crawl, so I killed the harness.
@jkeenan is this closable?
I'm closing this ticket because I will apparently never have enough memory on any machine to which I have access to configure with ASAN, build and test consistently. The other day I tried again. My machine slowed to a crawl during make and I had to reboot.
I spotted one problem with the discussion in pod/perlhacktips.pod about building with Address Sanitizer. I will file a new ticket for that.