OrangeC
OrangeC copied to clipboard
LADSoft
Investigate slow compiles
three files in the suite compile especially slowly:
sqlite3.c ilstream.cpp ilunstream.cpp
investigate why
@chuggafan reported the slowdown in sqlite3 is in the optimizer: specifically in the liveOut function.
ilstream and ilunstream should be easy enough to figure out, they have a large number of things that slow down compiles:
- Lots and lots of lambdas
- Lots and lots of template expansions from StreamBlock.
sqlite3 the tests were done in debug mode, YMMV for actual use-cases, I recommend testing against the Release mode MSVC for it.
I did some performance enhancements on the compiler; there were two flavors of enhancements. In the first, I supressed compiling a bunch of functions that don't really need to be compiled, and in the second I rewrote the function inliner to not be so dependent on using in-memory variables for function arguments. It is now able to use registers more often in for-loops for example... Together, that increased both the speed of compiles, and the speed of compiled code.
It is hard to be quantitative because on the one hand compiling the compiler itself takes longer now because the rewrites took us far away from simple lists and we use a lot of stl containers instead... but on the other there were a lot of other changes that improve that situation.
That said the compiles on appveyor seem several minutes faster on average than they did a few months ago.
At home I've noticed that when OCC is compiled with itself, it takes about 9 seconds to compile sqlite3.c... down from 14 before I started all these rewrites a few months ago. for comparison it takes about 7 seconds to compile sqlite3.c, when occ has been compiled with MSVC.
I watch builds a lot, I'm always running builds to verify that whatever changes I'm making aren't breaking things that used to work.
I can tell you from a qualitative standpoint things seem to be going by a lot faster than they did before the rewrites I've done over the last few months.
The c-testsuite is something I have especially been watching; the compiler compiled with MSVC used to be sluggish going through that. Compiling a little over 1 test/second. And compiled with OCC much worse. Now with MSVC it is noticeably faster (it compiles two to three tests a second), and it seems when compiled with OCC we are getting almost the performance we got out of it when compiled with MSVC just a few months ago.
Watching the builds of the tool chain, they are definitely faster as well...
i did specifically test with ilstream.cpp and ilunstream.cpp, they went from about 8 seconds down to about 2 seconds when I did part of the rewrite. Since then I've rewritten those files not to use lambdas though so it is a little quicker than that even
occil also seems faster, compiling the CC386 example use to take two-three seconds/file now it is down to one (when compiled with MSVC). There was a specific bottleneck with for loops that I isolated a couple of years ago; the rewrite would have improved that situation.
part of this was improving code generation... and part of the improvement to code generation was also able to clean up the intermediate code in such a way as to make the optimizer run a little faster... i took a look at the numbers at one point to verify that but unfortunately didn't capture the results.
All told I'm counting the recent work a success even though compiling the compiler itself seems a little slower because of the nature of the rewrites...
Right now the big problem with compiling the compiler is memory allocation by the way, it spends a good 30% of its time in new/delete (as measured in an MSVC build in release mode). Since some of the files in the compiler take 10-12 seconds to compile that is a big chunk of time (to be fair the files in question are around a half megabyte each and the worst offenders are just loaded with stl usage)... I tried briefly to improve that and did make some speedups, but there were problems with my approach to that and I've abandoned it for the moment...