llvm
Should clang-tidy support multiple threads?
clang-tidy is not multi-threaded
clang-tidy uses a single thread to run its checks. Looking at the generated profile data (--enable-check-profile), some of these checks take multiple minutes to finish. For large codebases with critical paths, clang-tidy is spending a lot of time analyzing individual source files which it could likely analyze faster by using multiple threads and running checks in parallel.
I see that run-clang-tidy.py script has a jobs (-j) flag, but this is simply spawning multiple clang-tidy processes (which most build systems & CIs will already be doing). I'm wondering if we should have an implementation of a -j flag for clang-tidy itself.
feature request
After the AST is parsed, it seems like running checks in parallel would make sense and be reasonable to implement. Is there a design limitation here that has prevented this feature, or has it simply not been proposed or implemented?
references
This is the only ticket I found that was relevant but did not address the absence of this feature: https://github.com/llvm/llvm-project/issues/55471
Many of the data structures in Clang is not set up to support multithreading and mutual exclusion. There was a significant effort put into specifically Clangd so it can be multithreaded. You can set up the Clang-Tidy drivers (either Clangd or CodeChecker) to run multiple instances of Clang-Tidy in parallel. It's much easier design-wise to run multiple processes (and let the ASTs and associated objects die when the process dies) than to deal with the added hassle of making sure all the data structures and execution model supports parallel execution...
There's no trivial way to implement this and given that spawning multiple processes is the defacto way to speed up processing of large code bases I don't see and reason to change this behaviour. There has been work to speed up clang-tidy by prevent checks from looking at certain header files which I feel is the correct way to go when trying to improve its performance.
Okay, thanks for clarifying that the implementation will be difficult. I agree that parallelizing clang-tidy processes and disabling checks is a good strategy to boost performance. We are already doing this currently, and I'm looking for ways to improve performance further.
For our particular case, we are using bazel as a build system and run clang-tidy as an aspect. The problem we observe is that due to dependencies and code generation, a critical path forms where clang-tidy processes must run serially. If the process takes multiple minutes, parallelizing at the individual check level would improve our process times and our overall analysis.
My current strategy will be to split our .clang-tidy file into equal weighted checks and run 2 processes for every source. As you said, we'll be using more processes to parallelize, but at least I'll be able to run checks in parallel and finish an individual source analysis faster.
