Very slow build because of system.hh header file bottleneck

[riastradh]

Just about every .cc file includes system.hh, and system.hh includes 84 header files including STL and boost.

Just the boost header files that are directly included alone, not counting STL header files or transitive inclusions, bring in half a megabyte of C++ code.

When I pass one of the smallest files in the build, mask.cc, through c++ -E to find the C preprocessor expansion, it turns out at 7.5 MB of C++ source code -- with no comments.

This makes builds very slow, because the compiler has to slog through megabytes of heavy-weight C++ declarations and templates that mostly aren't used by most of the .cc files. That, in turn, makes iterating development very slow -- takes minutes on my laptop to build ledger, which isn't a very large piece of software, even when I parallelize it to the maximum degree of my CPU (4-core/8-thread Kaby Lake 1.9 GHz).

It would be nice if each .cc file included only the header files it needs to speed up the build.

[afh]

I'd like to believe that precompiled headers should mitigate some the compile performance issues. Could it be that precompiled headers aren't working anymore / for you, @riastradh?

[riastradh]

It looks like, in ledger, precompiled headers are enabled only for debug builds:

https://github.com/ledger/ledger/blob/c679e3cbd5a44fa876c42b6912d9aa7729427c8f/src/CMakeLists.txt#L150

What I've been running locally in my ledger development tree, and in pkgsrc, has not been debug builds.

I tried a debug build, and it is spewing warning messages about pch files:

$ mkdir -p build/x86_64--netbsd9
$ cd build/x86_64--netbsd9
$ cmake -DUSE_PYTHON:BOOL=ON -DBUILD_DEBUG:BOOL=ON ../../master
...
$ make -j4
...
[  1%] Building CXX object src/CMakeFiles/libledger.dir/cmake_pch.hxx.gch
[  2%] Building CXX object src/CMakeFiles/libledger.dir/generate.cc.o
[  3%] Building CXX object src/CMakeFiles/libledger.dir/stats.cc.o
[  5%] Building CXX object src/CMakeFiles/libledger.dir/csv.cc.o
[  6%] Building CXX object src/CMakeFiles/libledger.dir/convert.cc.o
cc1plus: warning: /home/riastradh/misc/ledger/build/x86_64--netbsd9/src/CMakeFiles/libledger.dir/cmake_pch.hxx.gch: had text segment at different address
cc1plus: warning: /home/riastradh/misc/ledger/build/x86_64--netbsd9/src/CMakeFiles/libledger.dir/cmake_pch.hxx.gch: had text segment at different address
cc1plus: warning: /home/riastradh/misc/ledger/build/x86_64--netbsd9/src/CMakeFiles/libledger.dir/cmake_pch.hxx.gch: had text segment at different address
...

The debug build appears to be just as slow as the release build. Does this precompiled header mechanism rely on something kooky like disabling ASLR in the system?

(I don't think I've ever seen precompiled headers work, over decades of encountering them in various build systems, compilers, and operating systems. I still have -no-cpp-precomp wired into my fingers from two decades ago when it was needed to make just about any software build in an Apple environment...)

[afh]

Unfortunately I cannot answer your question about disabling ASLR, my expertise in the area is too little and the pch mechanism was added before I started contributing to ledger. Can you think of other ways to improve compile times to resolve this issue?

[riastradh]

Can you think of other ways to improve compile times to resolve this issue?

It would likely be much faster if each .cc file included only the header files it uses. For example, .cc files that don't handle pathnames or do file I/O probably don't need to include anything under boost/filesystem/.

Inside ledger, if a class A is declared in A.hh, and a class B is declared in B.hh with members or parameters that are pointers or references to A, you can also use forward declarations to reduce interdependencies between header files, which can help speed things up too:

/* B.hh */
class A;    // no need to #include "A.hh"
...
class B {
public:
  A *a;
...
};

[afh]

Could the ledger API documentation be helpful here in order to find which classes depend on/use which other classes?