OrangeC
OrangeC copied to clipboard
Adding a method to collect timing for OMAKE
So we already have timing for OCC, a logical next step so that we don't go over the 1 hour build mark is to have a command with OMAKE to collect the timing information from runs of the compiler into one place, with the file, so that we can get measurements of each file, and possibly see if a change causes affects on our compilation speed in the future.
I'm not suggesting we add this portion to AppVeyor btw, only that we have an option to collect this data in our back pocket, while there are known slow spots (looking at YOU sqlite3.c), being able to identify individual files to profile and see if a change is causing major problems down the line is something I think we can agree will be useful for at least Milestone 5.
This will also allow us to profile changes such as map vs unordered_map, etc. where we change what structure we use, or the change in the rebalance code on general applications (though that one is a bit more obvious for it being better).
The idea I have for this issue is that we run $(CC) with the timing option, then we take the results and send it up the chain with the file information attached, then we write it to a master file, from there, we can sort by overall execution time and then we can profile it in visual studio. We can even possibly collect information such as the number of lines vs the time taken, etc.
This will be a decently difficult thing, and I don't think this belongs in milestone 3 unless we absolutely have to do it for making sure we stay under an hour on our builds.
@chuggafan
agreed not to do it now.
ilstream.cpp and ilunstream.cpp also take inordinate time although I've made it better over the last couple of weeks with some tweaking... the stuff in the netlib directory takes a little longer than most of the other files in the project (at 3 seconds or so apiece).
sqlite3.c is a little different because there most of the time is in occopt whereas everything else overdoes it in occparse...
Meanwhile I've been working on low-hanging fruit.... made some minor adjustments to the memory management with some success (about 8-10% speedup based on the appveyor results). I've got a change to how inlining is handled on the table that looks like it might be a good improvement. It brought the compile of sqlite3.c down by several seconds, but I have to work the kinks out yet.
The really big problem as I see it is something you've mentioned a couple of times... occ allocates a lot of memory. Not even counting what is happening in memory.cpp, just the C++ allocations/deallocations are taking 10-15% of the compile time in occparse and in a lot of cases this is a major bottleneck. Many of the memory allocations happen during template instantiations... two things came out of that immediately, one we still need to completely rewrite the template engine to not have to reparse the source code during every instantiation and two that handling of types could be improved with some possibly good effects. But that is beyond the scope of what we should be working on in the short term...