pioneer icon indicating copy to clipboard operation
pioneer copied to clipboard

Refactor Header #include Knots

Open Web-eWorks opened this issue 2 years ago • 2 comments

This PR lowers compilation time by 15-20% on my development system by simply re-arranging our include files and not relying so heavily on the monolithic libs.h. As a result, the amount of time spent just textually including and parsing libs.h in a full rebuild has dropped by by over 75%.

Web-eWorks avatar Nov 22 '21 17:11 Web-eWorks

And yes, I borked the MSVC build myself, need to refactor the name of the new low-level math file...

Web-eWorks avatar Nov 22 '21 17:11 Web-eWorks

Update: I've not fully split utils.h and libs.h at this juncture, but I have made a significant reduction in the number of files unnecessarily including the latter. I'm happy with the compilation performance improvements at this juncture, and further changes should likely be coupled with a rearchitecture of utils.h and a separation of concerns between the logging functions and string comparison/manipulation functions. I'll leave that for a separate PR.

At this point I think this PR is ready for merge (once MSVC issues have been resolved). Feel free to review the PR, but there's very little substance to the changes other than reordering/refactoring #include directives.

EDIT: I will not be resolving the clang-format failures, as they're mostly spurious whitespace issues not introduced by this PR.

Web-eWorks avatar May 13 '22 23:05 Web-eWorks

I've updated this PR to current master and made further improvements. All numbers reported are compiled on a Ryzen 7 5800X.

Current master (68s wall-clock time):

**** Time summary:
Compilation (647 times):
  Parsing (frontend):          536.9 s
  Codegen & opts (backend):    330.0 s

Final result (46s wall-clock time):

**** Time summary:
Compilation (645 times):
  Parsing (frontend):          360.8 s
  Codegen & opts (backend):    189.2 s

Most of the codegen time improvement (~100s) came from removing the fmt::sprintf() call in the old Output functions and moving that inside a single compilation unit. The remainder of the performance improvement was reducing the number of transitory #includes of unneeded header files.

In total, this PR represents a 33% compilation time savings on my machine when compiled with 16 workers. The effect on CI builds will most likely be less, as they're comparatively less disk-bound and more cpu-bound.

Web-eWorks avatar Aug 26 '23 01:08 Web-eWorks

Additionally, it looks like CI compilation time went from ~16m to ~11m for Clang, and ~23m to ~16m for GCC.

Web-eWorks avatar Aug 26 '23 01:08 Web-eWorks

In total, this PR represents a 33% compilation time savings on my machine when compiled with 16 workers

Nice!

impaktor avatar Aug 26 '23 08:08 impaktor