cesium-native icon indicating copy to clipboard operation
cesium-native copied to clipboard

CI build improvements

Open jherico opened this issue 1 year ago • 1 comments

This PR ...

  • modifies the CMakeLists.txt configuration to take advantage of ccache/sccache when present.
  • refactors the CI build config to use a matrix approach
    • This will cause the Debug and RelWithDebugInfo to run in parallal as distinct jobs instead of serialized in a single job
    • It reduces the overall complexity of the CI config because now only two job definitions are required, Windows & Posix, vs five job definitions in the previous config. The Windows and Posix configs could probably be combined as well, but because of the sparse matrix desired for the Posix jobs it would probably not be worth it.

compiler caching

ccache/sccache are tools that will wrap the call from the build system (such as make or ninja) to the compiler and determine whether the file has already been compiled compared to the version in the cache. There are two kinds of cache hits...

  • when the file and none of its inputs have changed since the last build, called a direct hit
  • when the pre-processed output of the file is identical to an entry in the cache, called a pre-processed hit

Cache misses are built as normal but then the resulting object file is placed into the cache with the keys allowing the build to potentially skip that file in the future.

This can result in an order of magnitude drop in build times when the bulk of the files are unchanged, or effectively unchanged after preprocessing.

Fixes #814

jherico avatar Feb 21 '24 19:02 jherico

For comparison, here are the timings for a build where the compiler cache was empty...

image

And here are the timings for a build with the cache populated...

image

There are, generally, very large improvements, with the overall build time for an optimally used cache cut roughly in half. The limiting factor for getting it down even further is whatever seems to be going on here:

image

There's an excessive delay between the last compile line timestamp and the linking step. This delay only shows up in two places out of all the jobs... the RelWithDebInfo configs for both macos and gcc on ubuntu-latest. It doesn't happen with Clang on ubuntu and it doesn't happen on any debug config.

In both cases the compiler cache output looks roughly like this...

image

I'm unclear on what is causing the cache misses since the exact same code is being compiled as the prior execution. Assuming it is part of the compilation, I'm investigating the Catch2 functionality to see if there's any mechanism for fuzzing or randomization at build time of test inputs, which might contribute to defeating the compiler cache.

jherico avatar Feb 21 '24 22:02 jherico

Thanks @jherico, this is a nice improvement. Merging!

kring avatar May 10 '24 11:05 kring