Design multi-threaded build for Visual Studio scenarios

[AR-May]

In the current multi-process MSBuild setup, the devenv process hosts the MSBuild main node, while the majority of the build work is delegated to out-of-process MSBuild nodes. However, with the upcoming multi-threaded MSBuild model, all project builds will execute within the main node—i.e., inside the devenv process. This shift could potentially be problematic, because of possible crashes or for very large builds.

We need to design a mechanism that allows builds to run in a multi-threaded fashion outside of the devenv process. We should also allow Visual Studio to choose between single-process (multi-threaded) and multi-process execution modes.

[rainersigwald]

I just had a horrible/interesting thought about this. On very-high-core-count systems (like 64+), Windows and .NET often don't scale to having threads for all of the cores (see all our fun around https://github.com/dotnet/msbuild/pull/5625). We might lose some scaling on those types of machines if we actually try to be fully single-process. But we need to be able to push some work out of process anyway, for this VS scenario--we should consider if it'd be hard to have, for example, 2 processes each with 64 threads running, to satisfy /m:128.

[AR-May]

Once we can run in-proc and out-of proc thread nodes, I think it would not be that hard to make many processes each with many with thread nodes - we will need to update the scheduler to support spawning them in a certain configuration. One of issues that we will have - troubles with switching between the multi-threaded and multi-process modes when nodes are reused, but that should be resolvable. Should think more about it to see whether there would be other issues.

[surayya-MS]

Note: We might potentially need to serialize/deserialize Build.Request.ScheduledNodeId for VS scenarios. Build.Request.ScheduledNodeId was added in https://github.com/dotnet/msbuild/pull/12577 more in this comment https://github.com/dotnet/msbuild/pull/12577#discussion_r2393790524

[baronfel]

There are pretty big design questions that need to be answered in order to support multithreaded in MSBuild API-based consumers like VS:

• we probably need to separate the 'eval node' from the 'worker orchestration' node

  • for MSBuild Server mode those can be the same
  • for API consumers it may be useful for them to be separated
    • eval-host can be the host node
    • worker orchestration can then be an entirely separate MSBuild instance potentially
      • primary reason would be blast radius - something going wrong in a plug can't bring down devenv (or the host process)
      • opens up opportunity for using .NET MSBuild to a greater degree from VS in the future, without replacing the framework MSBuild entirely

• Need to know where VS would want to run thread-based worker nodes - either in devenv.exe or in a dedicated msbuild process.

  • There are reasons for both, need to consult with VS teams
    • @Michael-Eng for CPS
    • @JakeRadMSFT's dotnet/project-system
    • @itodirel for C++
    • @kamathac for VS Perf
  • Data/context to bring to this discussion:
    • perf counters from an exp insertion where we force-enable MT mode
      • should see lower process counts, fewer methods JITted, etc
    • perf counters from an exp insertion where we force-enable MT mode and have MSBuild Tasks enlightened
      • should show even clearer downward trends
    • future work that is enabled by separating eval node from work-orchestration node
    • possibility to make work-orchestration node .NET and keep eval node (devenv) as .NET Framework

• Any Task-work that arises specific for the VS environment