Binding redirects are not generated for transitive packages with version incremented with central package management transitive pinning

[vgriph]

Describe the bug

I'm not entirely sure if this is a bug with the .NET SDK or with the NuGet client, but I'm reporting it here to start with.

I've observed that when using enabling CentralPackageTransitivePinningEnabled, and using that to pin a package that is a dependency with include="Runtime,Build,Native,ContentFiles,Analyzers,BuildTransitive", binding redirects are not correctly generated for the updated transitive package, despite being needed to run the resulting application.

I've been trying to compare two different scenarios for building the same application. One in which I pin a version of a transitive dependency explicitly in the application project file, and one where the pinning is done via central package management.

In both cases the same dependencies are copied to the output directory, but only in the first case, the app.config gets assembly redirects generated for the dependency.

I've been comparing the msbuild logs of the two setups, and realized that, when the package is pinned using CentralPackageTransitivePinningEnabled, the transitive dependencies are not passed in the Assemblies parameter to the ResolveAssemblyReference, but when it is pinned by adding a PackageReference to the transitive dependency directly in the project, it is passed to the Assemblies parameter.

Digging further, it seems as if the transitive dependencies only appear as runtime items in project.assets.json, and the compile group is empty for the transitive dependencies. ( And that the ResovlePackageAssets target will load them only into RuntimeCopyLocalItems and not into ResolvedCompileFileDefinitions.) Without the CentralPackageTransitivePinningEnabled that makes sense, because without an explicit reference to the package I don't expect to be able to use types defined in it, and I wouldn't want an explicit reference added in my dll to that dependency, since I only link to the intermediate package.

However, the documentation for central package management specifies that CentralPackageTransitivePinningEnabled==true should create a top level package reference when needed. But it seems like this reference only exists during the version resolution phase, but not when the deciding compile time references in the project.assets.json or when deciding on compile time link assemblies.

When it comes to the actual generation of binding redirects, I think it would make sense if the assemblies passed to the ResolveAssemblyReference task would include the RuntimeCopyLocalItems, and not only the actual top level References resolved from the framework and the ResolvedCompileFileDefinitions

To Reproduce

I put together a minimal solution that illustrates the problem.

Download and extract the TestCPM.zip and restore and build the solution. The TestCPM.Broject will fail to generate binding redirects for System.Diagnostics.DiagnosticSource, but the TestCPM.Working will generate those redirects.

Exceptions (if any)

There are not exceptions when restoring or building the project. But when running the application, FileLoadException exceptions are thrown for the non-redirected assemblies. e.g

System.IO.FileLoadException : Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=7.0.0.2, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

Also looking at the output of the ResolveAssemblyReference it shows in result

Dependency "System.Diagnostics.DiagnosticSource, Version=7.0.0.2, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51". Could not resolve this reference. Could not locate the assembly "System.Diagnostics.DiagnosticSource, Version=7.0.0.2, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.

But that doesn't cause any build errors or warnings as far as I can tell.

Further technical details

.NET SDK:
 Version:           8.0.400
 Commit:            36fe6dda56
 Workload version:  8.0.400-manifests.56cd0383
 MSBuild version:   17.11.3+0c8610977

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.400\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
 [aspire]
   Installation Source: VS 17.11.35222.181
   Manifest Version:    8.1.0/8.0.100
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.1.0\WorkloadManifest.json
   Install Type:        FileBased


Host:
  Version:      8.0.8
  Architecture: x64
  Commit:       08338fcaa5

.NET SDKs installed:
  8.0.206 [C:\Program Files\dotnet\sdk]
  8.0.400 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.33 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.33 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

[vgriph]

I figured out a work-araound:

Adding the following two targets to Directory.build.targets:

  <Target Name="IncludeRuntimeAssetsInAssemblyResolution" BeforeTargets="ResolveAssemblyReferences" AfterTargets="ResolvePackageAssets">
    <ItemGroup>
      <Reference Include="@(RuntimeCopyLocalItems)" Exclude="@(Reference)">
        <FromRuntimeAssets>true</FromRuntimeAssets>
      </Reference>
    </ItemGroup>
  </Target>

  <Target Name="ExcludeRuntimeAssetsAfterAssemblyResolution" BeforeTargets="ResolveReferences" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <Reference Remove="@(Reference)" Condition="'%(Reference.FromRuntimeAssets)' == 'true'" />
    </ItemGroup>
  </Target>

[marcpopMSFT]

Thanks for the deep investigation. Since you pinpointed the version wasn't flowed through to ResolveAssemblyReference, I'm moving to MSBuild for now though this feels like it could require nuget/msbuild/sdk consultation.

[vgriph]

I've had to tweak my workaround a bit to get the correct result in complex solutions.

  <Target Name="IncludeRuntimeAssetsInAssemblyResolution" AfterTargets="ResolveLockFileReferences">
    <ItemGroup>
      <_RuntimeReferences Include="@(RuntimeCopyLocalItems)">
        <FromRuntimeAssets>true</FromRuntimeAssets>
        <HintPath>%(Identity)</HintPath>
      </_RuntimeReferences>
      <_RuntimeReferences Remove="@(Reference)" MatchOnMetadata="FileName" />
      <_RuntimeReferences Remove="@(Reference)" MatchOnMetadata="HintPath" />
    </ItemGroup>
    <ItemGroup>
      <Reference Include="@(_RuntimeReferences)" />
      <RuntimeCopyLocalItems Remove="@(_RuntimeReferences)" />
    </ItemGroup>
  </Target>

  <Target Name="ExcludeRuntimeAssetsAfterAssemblyResolution" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <RuntimeCopyLocalItems Include="@(Reference)" Condition="'%(Reference.FromRuntimeAssets)' == 'true' AND '%(Reference.CopyLocal)' == 'true'" />
      <ReferenceCopyLocalPaths Include="@(Reference)" Condition="'$(CopyLocalLockFileAssemblies)' == 'true' AND '%(Reference.FromRuntimeAssets)' == 'true' AND '%(Reference.CopyLocal)' == 'true'" />
      <Reference Remove="@(Reference)" Condition="'%(Reference.FromRuntimeAssets)' == 'true'" />
      <ReferencePath Remove="@(ReferencePath)" Condition="'%(ReferencePath.FromRuntimeAssets)' == 'true'" />
    </ItemGroup>
  </Target>

[SimaTian]

I'm currently busy with other stuff. Since I was handling other binding redirects related stuff, I will take a look at it eventually, but it might take a while. If anyone wishes to take over, I would consider it a fair game. However since it's prio 2 I reckon it can wait for me.

[SimaTian]

I've dug through this. Once again, thank you @vgriph for detailed explanation.

This was already mentioned, but the ResolveAssemblyReferences task doesn't get the data it would need to generate the binding redirects. The input is different from the get-go so the error lies from before the RAR resolution can begin.

As described in the ticket, the following is in the package.assets.json Working:

       "compile": {
          "lib/net471/Serilog.dll": {
            "related": ".xml"
          }
        },

Broken:

        "compile": {
          "lib/net471/_._": {
            "related": ".xml"
          }
        },

The transition from the package.assets.json to ResolveAssemblyReference assembly list is handled by dotnet - ResolvePackageAssets see here

The one thing that puzzles me is that this also happens when invoking MSBuild.exe (e.g. framework) so there is something more at play here.

For the next step, I will contact people from NuGet to ask for more information regarding the way the package.assets.json is generated.

[SimaTian]

I've talked with @rainersigwald and with @zivkan about the issue.

The final verdict is that this is a package bug - issue with a transitive reference that is avoided by doing manual pinning

The package currently states: <dependency id="Serilog" version="3.1.1" include="Runtime,Build,Native,ContentFiles,Analyzers,BuildTransitive" /> however it is missing the Compile in the include list.

This causes the RAR task to be called with an incorrect assembly list, resulting in insufficient Binding Redirects.