CodeCompass
CodeCompass copied to clipboard
Implementation of C++ metrics parallelization
Fixes #733
Added parallelization support to the C++ metrics parser plugin:
- Using
getFilterPathsQuery, we can now perform the same check as whatcc::util::isRootedUnderAnyOfused to do, but on the ODB-side. This is good because:- The number of records retrieved for manual processing by the client's handler is now drastically reduced.
- The
filePathmembers can now be removed from views that only needed them for this purpose. This further reduces the amount of data needed to be transferred per record as well.
- The
parallelCalcMetricmethod of the metrics parser now serves as the backbone of all parallel computations. - Existing metrics have been refactored to perform their outermost loop using the new parallelization mechanism.
Further considerations:
- [x] Buffering: Preload all records to already know the size on startup. (This is a memory overhead, but it might be worth it.)
- [x] Progress reporting: Building on the idea of buffering, report the current progress of each metric onto the log to increase interactivity. (Reporting at regular intervals in time and/or in steps.)
- [ ] Caching & Flushing: Collect all computed metrics into a single shared thread-safe container and persist everything at once at the end. This could eliminate the "one transaction for each worker" phenomenon currently observed. (Depending on the size of the overhead of opening and closing transactions, this could potentially give us a speed boost.)