Build taking too long / nearly a minute to make

[simonhf]

Build takes nearly a minute which seems too long for a relatively simple project like this with only a couple of files?

$ time make
+++ cleaning project
+++ compile: test/test_suite.cpp
... warnings ...
+++ linkink application to generate: bin/test_suite
+++ parsing symbols with nm to generate: bin/test_suite_nm.txt
+++ demangling symbols with c++filt to generate: bin/test_suite_cppfilt.txt
+++ creating size summary table with size to generate: bin/test_suite_size.txt
real    0m55.625s

$ gcc --version | egrep gcc
gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008

On a recent Ubuntu with recent gcc...

[ledvinap]

Catch testing framework is a bit slow to compile ...

[mpaland]

I think Petr is right, but 55 sec is really slow. I can't reproduce that (yet).

[simonhf]

Yep, it looks like the testing framework is perhaps overkill for what it's doing; nearly 15,000 LOC in one hpp file for only ~ 400 relatively simple tests which all have a similar structure:

$ wc -l catch.hpp test_suite.cpp 
 14934 catch.hpp
  1513 test_suite.cpp

Why have the monster hpp file? Wouldn't a couple of home grown #defines do just as well?

I like it that the ~ 400 tests are in one handy file, but I was expecting the test file to have more like ~ 400 lines (one for each test) and not 1,513 lines...

Further suggestion: Also -- assuming that the printf() library is designed to give the same output as the libc printf() -- what about the idea of running each test not only against the existing pre-defined expected string result, but also against the test result using the currently installed libc printf()? Seems like a low hanging fruit? :-)

[ledvinap]

@simonhf : https://github.com/ledvinap/printf/blob/inprogress/test/test_suite.cpp

[ledvinap]

@simonhf : It's quite usable quick tests and edge case detection, but IMO it's better to use well-defined test cases in long-term

[simonhf]

@mpaland I recently discovered Apline Linux which is so small you can just install and run it on the fly as a throw-away container :-) Advantage is that in theory it makes it easy to reproduce a known environment which reproduces an issue. If you have docker installed on your system then you should be able to run this one line to reproduce the very slow make; in this case taking 1m9s:

$ docker run -it --rm -v `pwd`:/extern alpine:3.11.0 sh -c "apk update ; apk add bash git make g++ ; cd /extern/ ; rm -rf printf/ ; git clone https://github.com/mpaland/printf.git ; cd printf ; time make"
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
v3.11.3-73-g095aa9b9d4 [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.3-72-g77ec45c4dc [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
OK: 11266 distinct packages available
(1/25) Installing ncurses-terminfo-base (6.1_p20191130-r0)
(2/25) Installing ncurses-terminfo (6.1_p20191130-r0)
(3/25) Installing ncurses-libs (6.1_p20191130-r0)
(4/25) Installing readline (8.0.1-r0)
(5/25) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
(6/25) Installing libgcc (9.2.0-r3)
(7/25) Installing libstdc++ (9.2.0-r3)
(8/25) Installing binutils (2.33.1-r0)
(9/25) Installing gmp (6.1.2-r1)
(10/25) Installing isl (0.18-r0)
(11/25) Installing libgomp (9.2.0-r3)
(12/25) Installing libatomic (9.2.0-r3)
(13/25) Installing mpfr4 (4.0.2-r1)
(14/25) Installing mpc1 (1.1.0-r1)
(15/25) Installing gcc (9.2.0-r3)
(16/25) Installing musl-dev (1.1.24-r0)
(17/25) Installing libc-dev (0.7.2-r0)
(18/25) Installing g++ (9.2.0-r3)
(19/25) Installing ca-certificates (20191127-r1)
(20/25) Installing nghttp2-libs (1.40.0-r0)
(21/25) Installing libcurl (7.67.0-r0)
(22/25) Installing expat (2.2.9-r1)
(23/25) Installing pcre2 (10.34-r1)
(24/25) Installing git (2.24.1-r0)
(25/25) Installing make (4.2.1-r2)
Executing busybox-1.31.1-r8.trigger
Executing ca-certificates-20191127-r1.trigger
OK: 196 MiB in 39 packages
Cloning into 'printf'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 526 (delta 0), reused 3 (delta 0), pack-reused 523
Receiving objects: 100% (526/526), 392.80 KiB | 1.53 MiB/s, done.
Resolving deltas: 100% (312/312), done.
+++ cleaning project
+++ compile: test/test_suite.cpp
... warnings ...
+++ linkink application to generate: bin/test_suite
+++ parsing symbols with nm to generate: bin/test_suite_nm.txt
+++ demangling symbols with c++filt to generate: bin/test_suite_cppfilt.txt
+++ creating size summary table with size to generate: bin/test_suite_size.txt
real    1m 9.40s
user    1m 6.47s
sys     0m 2.92s

This docker one-liner downloads the ~ 5 MB Alpine Linux version 3.11 container on-the-fly, and the installs a couple of needed packages, clones mpaland printf, and builds it while timing the build... all on-the-fly... ;-)

[simonhf]

BTW I used the above example as illustration in this Quora post [1] :-) [1] https://www.quora.com/Why-is-Alpine-Linux-so-popular-as-base-Docker-image/answer/Simon-Hardy-Francis

[eyalroz]

catch2 is notoriously slow to build. doctest is faster, and often much faster. Maybe there are even faster options to consider (e.g. C-only unit testing frameworks? Ones with less features?)

Still, since the library itself is fine, I'd say this is very low-priority.