fsharp
fsharp copied to clipboard
Investigate why build processes started consuming much more memory
After moving to the .NET 7 for builds, dotnet/fsc (via buildscripts) consumes much more memory (20+Gb).
This affects our CI too, apparently (build times out sometimes).
cc @KevinRansom
GC regions probably?
FWIW I'm getting this pretty consistently, seeing >=20GB memory used and lots of disk swap happening, making the build very slow.
These are the last projects that build mostly fine:
FSharp.Build.UnitTests -> C:\projekty\fsharp\fsharp\artifacts\bin\FSharp.Build.UnitTests\Debug\net472\FSharp.Build.UnitTests.dll
FSharp.Build.UnitTests -> C:\projekty\fsharp\fsharp\artifacts\bin\FSharp.Build.UnitTests\Debug\net6.0\FSharp.Build.UnitTests.dll
TestTP -> C:\projekty\fsharp\fsharp\artifacts\bin\TestTP\Debug\netstandard2.0\TestTP.dll
Then it takes ~5 minutes for these two to appear:
FSharp.Core.UnitTests -> C:\projekty\fsharp\fsharp\artifacts\bin\FSharp.Core.UnitTests\Debug\net472\FSharp.Core.UnitTests.dll
FSharp.Core.UnitTests -> C:\projekty\fsharp\fsharp\artifacts\bin\FSharp.Core.UnitTests\Debug\net6.0\FSharp.Core.UnitTests.dll
(there are more projects in the build).
I ran DotMemory for ./build.cmd -noVisualStudio
, here is a snapshot from one of the processes.
It's quite clear that CollectionTracing.Entry
is to blame:
Ok, confirmed that the above was an issue with my local environment and doesn't apply more widely, so please ignore my previous claims of the build taking 20GB+.
Details:
I was doing some tests a few weeks ago with tracing collection allocations. The code was retaining every allocation and a huge amount of memory.
I since got rid of the code but the built FSharp.Core.dll
artifact remained in the artifacts/Bootstrap
directory which I never cleaned since performing those tests.
I could not reproduce high memory consumption in build on .NET 7p6. If someone experiences this again please report it.