Building C::Blocks takes a long time

[tsee]

I just built C::Blocks on a rather vanilla Linux (4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux) using the stock distribution perl 5.22.1.

Building C::Blocks takes "forever" in the Build stage because it falls into brute_force_screen in PerlAPI.xs.PL

tsee@l4tsee:~/perl/C-Blocks$ time ./Build 
Building C-Blocks
Creating lib/C/Blocks/PerlAPI.xs
tcc: error: undefined symbol 'PL_memory_wrap'
tcc: error: undefined symbol 'Perl_croak_nocontext'
^^^ Ignoring likely erroneous undefined symbol warning/error
Using executable screen to check 2755 identifiers

...

Using DynaLoader screen to check 2411 identifiers
Beginning brute-force screening
Using DynaLoader screen to check 2411 identifiers
ccache gcc -I/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/include -I. -I/usr/lib/x86_64-linux-gnu/perl/5.22/CORE -DVERSION="0.42" -DXS_VERSION="0.42" -fPIC -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -o lib/C/Blocks.o lib/C/Blocks.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/C/Blocks/Blocks.bs')
x86_64-linux-gnu-gcc -shared -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/C/Blocks/Blocks.so lib/C/Blocks.o -L/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/lib -ltcc
ccache gcc -I/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/include -I. -I/usr/lib/x86_64-linux-gnu/perl/5.22/CORE -DXS_VERSION="0.42" -DVERSION="0.42" -fPIC -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -o lib/C/Blocks/PerlAPI.o lib/C/Blocks/PerlAPI.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/C/Blocks/PerlAPI/PerlAPI.bs')
x86_64-linux-gnu-gcc -shared -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/C/Blocks/PerlAPI/PerlAPI.so lib/C/Blocks/PerlAPI.o -L/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/lib -ltcc

real    6m59.862s
user    6m59.248s
sys     0m13.268s

(I think that the "starting brute force scan" showing up late in the output is a STDOUT/STDERR mixing or a PerlIO buffering effect.)

The bit that I found surprising was that this uses the system compiler rather than TCC, so there's clearly something about the purpose of this that I haven't intuitively understood (didn't dig).

HTH, Steffen

[tsee]

So this changes significantly due to:

#####################
# executable_screen #
#####################
# Given a list of identifiers, puts all of them in a C file and
# processes the output of compiler and linker errors to remove obvious
# bad ones. This fails to link on Debian systems unless the system has
# libperl-dev, or this is being installed on a custom perl (or perlbrew)
# environment. Brute-force will work, but is much slower.

After installing libperl-dev, ./Build is 60x faster:

~/perl/C-Blocks$ time ./Build 
Building C-Blocks
Creating lib/C/Blocks/PerlAPI.xs
tcc: error: undefined symbol 'PL_memory_wrap'
tcc: error: undefined symbol 'Perl_croak_nocontext'
^^^ Ignoring likely erroneous undefined symbol warning/error
Using executable screen to check 2755 identifiers
Using executable screen to check 2670 identifiers
Using executable screen to check 2412 identifiers
Using DynaLoader screen to check 2412 identifiers
Using DynaLoader screen to check 2411 identifiers
Beginning brute-force screening
Using DynaLoader screen to check 2411 identifiers
ccache gcc -I/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/include -I. -I/usr/lib/x86_64-linux-gnu/perl/5.22/CORE -DVERSION="0.42" -DXS_VERSION="0.42" -fPIC -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -o lib/C/Blocks.o lib/C/Blocks.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/C/Blocks/Blocks.bs')
x86_64-linux-gnu-gcc -shared -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/C/Blocks/Blocks.so lib/C/Blocks.o -L/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/lib -ltcc
ccache gcc -I/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/include -I. -I/usr/lib/x86_64-linux-gnu/perl/5.22/CORE -DXS_VERSION="0.42" -DVERSION="0.42" -fPIC -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -o lib/C/Blocks/PerlAPI.o lib/C/Blocks/PerlAPI.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/C/Blocks/PerlAPI/PerlAPI.bs')
x86_64-linux-gnu-gcc -shared -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/C/Blocks/PerlAPI/PerlAPI.so lib/C/Blocks/PerlAPI.o -L/usr/local/share/perl/5.22.1/auto/share/dist/Alien-TinyCCx/lib -ltcc

real    0m6.482s
user    0m6.156s
sys     0m0.384s

Is it worth trying to add a warning to that effect ("you don't seem to have your perl headers installed, if you're running a Linux distribution supplied perl, consider installing the perl headers package (eg. libperl-dev on debian derivatives") or so?

--Steffen

[run4flat]

I know that nobody reads the README, but this is mentioned there: https://github.com/run4flat/C-Blocks#build-notes

I think your idea about a warning is a very good one. Perhaps I could use something like IO::Prompter to intelligently issue a warning and ask if users want to bail and install the Perl development files on their system.

[tsee]

Personal nit: I'd stay away from IO::Prompter and anything fancy like that. In particular, any user interaction at Build.PL/configure time needs to automatically degrade to "just choose defaults" or "don't bother" if in a fully automated build environment (such as CPAN testers). So I'd probably just go for warn() with a good error message. If it's an interactive session, then the user will see the warning because then the build is actually quite slow anyway...

[run4flat]

Hmm, know of any handy incantation for detecting if the appropriate header files are installed? Is Devel::CheckLib reliable enough here? Or, perhaps trying to run the executable screen without any symbols would give a reliable result?

[tsee]

I don't understand the code here very well, but indeed I was assuming that since its purpose is to try and compile things and it bombs if it doesn't find stuff, that would be the easiest way to tell. :)