msbuild icon indicating copy to clipboard operation
msbuild copied to clipboard
verified publisher icon dotnet

Honor BuildInParallel

Open vindicatorr opened this issue 3 weeks ago • 5 comments

Commit 58047eaec542dec6081cd0393dc710f3f5d739be

My old system is very resource-constrained so I normally need to compile with the equivalent of --jobs 1 --load 2.0.

When building Roslyn, I had htop open because my system hung in a previous attempt (needed magic+sysrq+f (oom)), and saw all 4 cores were maxed and memory quickly climbed for the /opt/dotnet/dotnet exec /opt/dotnet/sdk/9.0.306/Roslyn/bincore/VBCSCompiler.dll -pipename:<pipeName> process.

Yes, "9.0". I made some small alterations to try to build with what I have (including rolling-back the arcade version). I've successfully done this in the past with omnisharp. It just takes time for me to figure out. But the "parallel" aspect is unrelated.

I looked anywhere I could for BuildInParallel that wasn't explicitly set to false, and forced it (eg src/Features/CSharpTest/Microsoft.CodeAnalysis.CSharp.Features.UnitTests.csproj and src/LanguageServer/Microsoft.CodeAnalysis.LanguageServer/PackAllRids.targets).

But even then, my command:

$ dotnet build \
--no-restore \
-verbosity:diag \
-maxCpuCount:1 \
-property:RestoreBuildInParallel=false \
-property:BuildInParallel=false \
-property:Configuration=Release \
-property:Platform="Any CPU" \
-property:TargetFramework=net9.0 \
-property:TargetRuntime=linux-x64 \
-property:DotNetBuildSourceOnly=true \
-property:DotNetBuildFromVMR=true \
-property:TargetRid=linux-x64 \
-property:RepoRoot=${PWD}/ \
-property:TreatWarningsAsErrors=false \
Compilers.slnf \
2>&1 | tee build.log

still resulted in:

<...>
/<pathTo>/roslyn/src/NuGet/Microsoft.CodeAnalysis.BuildClient.Package/Microsoft.CodeAnalysis.BuildClient.Package.csproj
                  AdditionalProperties=Configuration=Release; Platform=AnyCPU
                  Configuration=Release
                  Platform=AnyCPU
                  SkipNonexistentProjects=False
                  ToolsVersion= (TaskId:3)
  Task Parameter:BuildInParallel=True (TaskId:3)
  Global Properties: (TaskId:3)
    BuildingSolutionFile=true (TaskId:3)
    CurrentSolutionConfigurationContents=<SolutionConfiguration>
    <ProjectConfiguration Project="{600AF682-E097-407B-AD85-EE3CED37E680}" AbsolutePath="/<pathTo>/roslyn/src/Deployment/RoslynDeployment.csproj" BuildProjectInSolution="True">Release|AnyCPU</ProjectConfiguration>
<...>

Note BuildingSolutionFile=true for that task.

While the system resources were climbing for the process, I froze it to see the last stuff outputted was:

<...other *.cs exceeding beyond the scrollback buffer>
/<pathTo>/src/Compilers/CSharp/CSharpAnalyzerDriver/CSharpDeclarationComputer.cs "/<pathTo>/artifacts/obj/Microsoft.CodeAnalysis.CSharp/Release/net9.0/.NETCoreApp,Version=v9.0.AssemblyAttributes.cs" /<pathTo>/artifacts/obj/Microsoft.CodeAnalysis.CSharp/Release/net9.0/Microsoft.CodeAnalysis.CSharp.InternalsVisibleTo.cs /<pathTo>/artifacts/obj/Microsoft.CodeAnalysis.CSharp/Release/net9.0/Microsoft.CodeAnalysis.CSharp.CSharpResources.cs /<pathTo>/artifacts/obj/Microsoft.CodeAnalysis.CSharp/Release/net9.0/Microsoft.CodeAnalysis.CSharp.AssemblyInfo.cs /warnaserror+:NU1605,SYSLIB0011 /warnaserror-:IDE0055 (TaskId:307)
  Assembly loaded during TaskRun (Microsoft.CodeAnalysis.BuildTasks.Csc): System.Security.Principal.Windows, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /opt/dotnet/shared/Microsoft.NETCore.App/9.0.10/System.Security.Principal.Windows.dll, MVID: d7ae0db5-c7a2-4ffb-94b9-b59a3271f1fc, AssemblyLoadContext: Default) (TaskId:307)
  Assembly loaded during TaskRun (Microsoft.CodeAnalysis.BuildTasks.Csc): System.Security.Claims, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /opt/dotnet/shared/Microsoft.NETCore.App/9.0.10/System.Security.Claims.dll, MVID: 57921f00-3884-44e5-a77c-29ab174756fa, AssemblyLoadContext: Default) (TaskId:307)
  Assembly loaded during TaskRun (Microsoft.CodeAnalysis.BuildTasks.Csc): System.IO.Pipes, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /opt/dotnet/shared/Microsoft.NETCore.App/9.0.10/System.IO.Pipes.dll, MVID: b7b52840-70ad-48cd-b397-7a554b891b4f, AssemblyLoadContext: Default) (TaskId:307)
  Assembly loaded during TaskRun (Microsoft.CodeAnalysis.BuildTasks.Csc): System.Diagnostics.StackTrace, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /opt/dotnet/shared/Microsoft.NETCore.App/9.0.10/System.Diagnostics.StackTrace.dll, MVID: 6bcf773e-a3c7-4849-924e-d43e57ae6a0e, AssemblyLoadContext: Default) (TaskId:307)

So is the issue with VBCSCompiler.dll not receiving any parallel option? A quick ildasm glance didn't show any such capability.

vindicatorr avatar Dec 06 '25 22:12 vindicatorr

Can you provide a binlog from this build? Thanks.

jjonescz avatar Dec 08 '25 13:12 jjonescz

You request a binlog, but they aren't allowed to be posted here. File type .binlog not supported. See the [documentation](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files) for supported file types.

I follow the URI and see the option: You can share your logs with us anonymously via MS developer community. making me think this is the preferred method.

I follow that URI and see another form that insists on a minimum title length (I was just going to copy/paste my title here), and a minimum body length (I was going to copy/paste this OP URI).

I then see the cookie-trail Home  .NET  Report a problem, making me think it's a general report area, and wondering how it'll get to you.

Maybe clarify in the docs how to procede, or add binlogs as an accepted file. Since I see the file appears to be gzipped, I could always just rename it, but again the doc you linked says nothing about that.

vindicatorr avatar Dec 08 '25 20:12 vindicatorr

Your Developer Community post itself is publicly visible, while the file attachments to it are private to you and Microsoft. Once you have created the post, you can link to it here.

RikkiGibson avatar Dec 08 '25 20:12 RikkiGibson

Here's hoping this was done correctly: https://developercommunity.microsoft.com/t/Roslyn---Honor-BuildInParallel/11012884

EDIT0: I want to add a side-note that it seems the dotnet exec pipe command remains as a process after the build ends. It did the same for the MSBuildStructuredLog project too.

vindicatorr avatar Dec 08 '25 22:12 vindicatorr

Thanks, I see a single BuildInParallel = True for the .sln.metaproj which is generated by MSBuild whenever you build a solution.

  <Target Name="Build" Outputs="@(CollectedBuildOutput)">
    <MSBuild BuildInParallel="True" Properties="BuildingSolutionFile=true; CurrentSolutionConfigurationContents=$(CurrentSolutionConfigurationContents); SolutionDir=$(SolutionDir); SolutionExt=$(SolutionExt); SolutionFileName=$(SolutionFileName); SolutionName=$(SolutionName); SolutionPath=$(SolutionPath)" Projects="@(ProjectReference)">
      <Output TaskParameter="TargetOutputs" ItemName="CollectedBuildOutput" />
    </MSBuild>
  </Target>

It seems to be hardcoded here (and in other places in the same file):

https://github.com/dotnet/msbuild/blob/60df124f65e3ad02433097bd7067f9c030fa1af1/src/Build/Construction/Solution/SolutionProjectGenerator.cs#L2017

Moving to msbuild repo.

jjonescz avatar Dec 09 '25 11:12 jjonescz