dotnet
New SDL requirement: Enable deterministic builds
A new SDL requirement recently came online requiring us to produce deterministic builds for our managed assemblies. This caused a bunch of alerts to light up across Arcade's internal repo health dashboard.
In theory this should be a simple matter of adding <Deterministic>true</Deterministic> to the shared .props file under our source directory.
More info:
- https://learn.microsoft.com/dotnet/csharp/language-reference/compiler-options/code-generation#deterministic
- https://aka.ms/liquid?share=17264 (MSFT internal)
Figured we could start in this repo and squash the alerts here, then track the binaries flowing to other repos and see if the alerts start disappearing.
A tangential concern about nuget packages, not the managed assemblies:
I have found that the MSBuild Tasks implemented in this repo (such as https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.NuGetRepack/tasks/src/ReplacePackageParts.cs and https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.NuGetRepack/tasks/src/UpdatePackageVersionTask.cs) mutate existing nuget packages as part of the VMR build. The mutation changes are not deterministic. In fact, when run against a nuget package already built in <Deterministic>true</Deterministic> mode, they actually undo the deterministic-ness.
@omajid Do you have an example of the diffs that are showing up?
Btw, good to see you found https://github.com/NuGet/Home/issues/8601. This issue overall was blaring warning lights from long ago deep in my head and I had to go hunting for it.
I built a custom SDK which enables deterministic mode. Then I used it to build the VMR in source-build mode twice. Running diffoscope against the two SDKs shows things like:
I am working on a fix (or hack, I guess) for this. WIP (doesn't even build) is here: https://github.com/omajid/dotnet-dotnet/commits/reproducible/
Actually most of the blame for Microsoft.DotNet.ILCompiler.nupkg non-determinism is the arcade/src/Microsoft.DotNet.Build.Tasks.Packaging/src/NuGetPack.cs task.
@GrabYourPitchforks @mmitche I'm going to reach out to the two of you about deterministic pack.
Any chance you can loop me in too? I would like to understand any constraints/limitations that are in play. Thanks!
In theory this should be a simple matter of adding
<Deterministic>true</Deterministic>to the shared .props file under our source directory.
FYI, I have been told that this is already the default: https://github.com/dotnet/dotnet/pull/1618#issuecomment-3117400680