Increase parallelism when using clang-build.ps1 with -parallel switch

[GerrietReents]

Thank you again for your great work! I have started to use your build script clang-build.ps1 on our build servers, too.

I have discovered, that using the -parallel switch the projects were built in sequence and only the source files were built in parallel. This is fine for projects with many files. My projects often do not contain as many files as the build server has cores, so parallelization is not that good.

Is it possible to increase parallelism by building projects in parallel as well?

Maybe using powershell workflows and parallel foreach would be a possible solution.

[cpp-red-lion]

Hi,

Interesting suggestion.
We thought about it a few months back, but didn't pursue it then...

I seem to remember some issues we identified with this workflow (at least in our codebase) , but I can't seem to remember them, now 🤔

Maybe we should reconsider it.

[hero101111]

Hello,

Sadly, this is a limitation of the current way we do parallelism (at file level). We've looked into building projects in parallel and it is poses a number of challenges:

• grabbing execution log in a synchronous manner from asynchronous project workers.
• workflows/parallel-foreach impose restrictions to PS code that are not really compatible with the way ClangPowerTools' clang-build.ps1 is built.

We've been able to avoid these problems with the file parallelism approach by keeping things as serial as possible. The parallel worker jobs are launched at the latest possible time, when everything has been computed and all that's left to do is call clang++/clang-tidy. This cannot be done at project-level because every project needs to compute its own settings, flags, etc.

Sadly we don't currently have the time for developing this. Outside help would be much appreciated.

Regards, Gabriel