Migration bundle creation fails on DevOps, does not authenticate against private NuGet feed

[ghost]

trafficstars

Running dotnet ef migrations bundle in a DevOps pipeline task fails, because the projects consume packages from a private NuGet feed.

The repo contains a NuGet.config file which points to the feed. The feed's credentials are declared in Service Connections in DevOps. The DevOps pipeline has a .NET Core Restore step which specifies the path to NuGet.config and the aforementioned credentials.

The Restore step runs successfully and is able to fetch packages from that feed, but the ef migrations bundle command doesn't pick up the credentials and I don't see a way to configure it to do so.

EDIT - after some further trial and error, if the NuGet.config file contains the following (redacted):

<packageSourceCredentials>
  <OurPrivateNuGet>
    <add key="Username" value="ClearTextUsername" />
    <add key="ClearTextPassword" value="ClearTextPassword" />
  </OurPrivateNuGet>
</packageSourceCredentials>

then the migration bundle builds ok. But if NuGet.config contains %NuGetUsername% and %NuGetPassword% environment variables, these do not appear to be substituted as expected by the bundler. The environment variables are defined in the Variables tab in the DevOps pipeline.

Is it maybe a NuGet issue? I can work around it by using a PowerShell task to insert credentials into NuGet.config from secured environment variables.

Building bundle...
dotnet publish --runtime win10-x64 --output C:\Users\VssAdministrator\AppData\Local\Temp\o1ilebbo.deb\publish --no-self-contained --configuration Release
Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Retrying 'FindPackagesByIdAsyncCore' for source '[https://***.azurewebsites.net/nuget/FindPackagesById()?id='runtime.any.System.Runtime'&semVerLevel=2.0.0'.](https://%2A%2A%2A.azurewebsites.net/nuget/FindPackagesById()?id=%27runtime.any.System.Runtime%27&semVerLevel=2.0.0%27.)
Response status code does not indicate success: 401 (Unauthorized).

Full logs: BundleLogs.txt

Using the --no-build argument doesn't change the outcome, it still invokes the publish command as shown above and fails in the same way.

EF Core version: 6.0.2 Database provider: Microsoft.EntityFrameworkCore.SqlServer 6.0.2 with Microsoft.Data.SqlClient 4.1.0 Target framework: .NET 6.0 Operating system: windows-2022 (DevOps agent) IDE: Visual Studio 2022

DevOps pipeline:

1. Restore step:

2. Build

3. Publish

4. Install EF global tool

5. Bundle:

6. Publish web project artifact

7. Publish efbundle.exe artifact

[ajcvickers]

@bricelam I think I understand this a bit better now. Let's discuss in our 1:1.

[ajcvickers]

Note from triage: the issue here is that building the bundle performs a restore, which does not use the configuration set up for the pipeline restore step. It's possible that we may not need to do a restore here, or could ignore errors--see also #27022. Putting this in 7.0 to investigate if there is anything simple we can do that would be better, but if this turns out to be complicated we may need to punt for 7.0.

[srmagura]

@bricelam @ajcvickers This is a blocking issue for my team because it is failing our CI pipeline ~60% of the time. It had been working flawlessly for around 3 months so I'm not sure what changed. Perhaps a change introduced in 6.0.2.

I see the OP has suggested a workaround of adding <packageSourceCredentials> to nuget.config. I may try that, but it's still not ideal because (1) you then have credentials in your Git repo and (2) it's a confusing extra step that doesn't match how "service connections" normally work in Azure Pipelines.

There's still 7 months until EF 7 is released. Would it be possible to prioritize this issue? Let me know if I can help by providing a minimal repro.

Side notes:

1. I don't understand why a restore still occurs even if --no-build is passed.
2. We are using the modern YAML-based pipelines, but otherwise our issue appears to be the exact same as the OP's.
3. Our pipeline and private NuGet feed are in the same DevOps organization... wondering if that's why it worked until recently.

[ghost]

@srmagura You shouldn't need credentials in the Git repo. I've only got the credentials stored as secured environment variables in the Azure pipeline, and use a PowerShell pipeline task to create the nuget.config file on the fly, with those credentials being injected into the file.

[srmagura]

@danielgreen-nes You are correct & thanks for the tip! That extra step just adds a tiny bit of complexity to the pipeline which I would like to avoid.

This is no longer a blocking issue for my team. I just added the <packageSourceCredentials> to nuget.config, since we're not worried about anyone "stealing" our internal NuGet packages.

[voroninp]

Something weird is happening indeed

Building bundle... dotnet publish --runtime debian.11-x64 --output /tmp/0y0q0v0k.xfw/publish --no-self-contained --configuration Release

but then I see this warning:

/usr/share/dotnet/sdk/6.0.300/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(1114,5): warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used [/src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj]

Also slightly above in the log:

Using project '/src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj'.
Using startup project '/src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj'.
Writing '/src/DataAccessLayer.Migrations.PgSql/obj/DataAccessLayer.Migrations.PgSql.csproj.EntityFrameworkCore.targets'...
dotnet msbuild /target:GetEFProjectMetadata /property:EFProjectMetadataFile=/tmp/tmpjueNv7.tmp /verbosity:quiet /nologo /src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj
Writing '/src/DataAccessLayer.Migrations.PgSql/obj/DataAccessLayer.Migrations.PgSql.csproj.EntityFrameworkCore.targets'...
dotnet msbuild /target:GetEFProjectMetadata /property:EFProjectMetadataFile=/tmp/tmp8gGDg5.tmp;Configuration=Release /verbosity:quiet /nologo /src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj
dotnet exec --depsfile /src/DataAccessLayer.Migrations.PgSql/bin/Release/net6.0/CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql.deps.json --additionalprobingpath /root/.nuget/packages --runtimeconfig /src/DataAccessLayer.Migrations.PgSql/bin/Release/net6.0/CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql.runtimeconfig.json /root/.nuget/packages/dotnet-ef/6.0.5/tools/netcoreapp3.1/any/tools/netcoreapp2.0/any/ef.dll migrations bundle --force -o /app/migrate.exe --assembly /src/DataAccessLayer.Migrations.PgSql/bin/Release/net6.0/CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql.dll --project /src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj --startup-assembly /src/DataAccessLayer.Migrations.PgSql/bin/Release/net6.0/CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql.dll --startup-project /src/DataAccessLayer.Migrations.PgSql/DataAccessLayer.Migrations.PgSql.csproj --project-dir /src/DataAccessLayer.Migrations.PgSql/ --root-namespace CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql --language C# --framework net6.0 --configuration Release --nullable --working-dir /src --verbose
Using assembly 'CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql'.
Using startup assembly 'CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql'.
Using application base '/src/DataAccessLayer.Migrations.PgSql/bin/Release/net6.0'.
Using working directory '/src/DataAccessLayer.Migrations.PgSql'.
Using root namespace 'CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql'.
Using project directory '/src/DataAccessLayer.Migrations.PgSql/'.
Remaining arguments: .
Finding DbContext classes...
Finding IDesignTimeDbContextFactory implementations...
Found IDesignTimeDbContextFactory implementation 'PgSqlFulfillmentContextFactory'.
Found DbContext 'OrderFulfillmentDbContext'.
Finding application service provider in assembly 'CarNext.OrderFulfillment.DataAccessLayer.Migrations.PgSql'...
Finding Microsoft.Extensions.Hosting service provider...
No static method 'CreateHostBuilder(string[])' was found on class 'Program'.
No application service provider was found.
Finding DbContext classes in the project...
Using DbContext factory 'PgSqlFulfillmentContextFactory'.
Using context 'OrderFulfillmentDbContext'.
'OrderFulfillmentDbContext' disposed.
Building bundle...
dotnet publish --runtime debian.11-x64 --output /tmp/0y0q0v0k.xfw/publish --no-self-contained --configuration Release

They way you start the build somehow ignores Directory.build.props file where I define some MSBuild properties:

<SolutionRootFolder>$([MsBuild]::GetDirectoryNameOfFileAbove('.', 'CarNext.OrderFulfillment.sln'))</SolutionRootFolder>

So I see the warning:

/usr/share/dotnet/sdk/6.0.300/Microsoft.Common.CurrentVersion.targets(2066,5): warning : The referenced project '/Analyzers/GlobalAnalyzers/GlobalAnalyzers.csproj' does not exist. [/src/Utilities/Utilities.csproj]

<ItemGroup>
    <ProjectReference
        Include="$(SolutionRootFolder)\Analyzers\GlobalAnalyzers\GlobalAnalyzers.csproj"
        Condition="!$(MSBuildProjectDirectoryNoRoot.Contains('Analyzers'))">
        <PrivateAssets>all</PrivateAssets>
        <ReferenceOutputAssembly>false</ReferenceOutputAssembly>
        <OutputItemType>Analyzer</OutputItemType>
    </ProjectReference>
</ItemGroup>

[ErikEJ]

The warning is just a SDK bug..

[mitankaradev]

Could this task help as a workaround? - task: NuGetAuthenticate@1

[danielgreen]

@ajcvickers @bricelam @roji @ErikEJ Is there any indication when we might see a fix for this? It's not great for developers to release and promote this tooling when it's broken for such a common scenario, and provide no updates or workarounds

[mikanygTC]

Could this task help as a workaround? - task: NuGetAuthenticate@1

Confirmed working

[jdotolsson]

Could this task help as a workaround? - task: NuGetAuthenticate@1

Confirmed working

Could you post a example of the pipeline?

I currently is experiencing this issue myself, but I cannot get the nuget.config workaround to work.

EDIT: Figured it out myself:

steps:
  - checkout: self

  - task: DotNetCoreCLI@2
    displayName: Restore
    inputs:
      command: 'restore'
      feedsToUse: 'config'
      nugetConfigPath: './nuget.config'
      projects: '**/*.csproj'

  - task: DotNetCoreCLI@2
    displayName: Build
    inputs:
      command: 'build'
      projects: '**/*.csproj'
      arguments: '--no-restore'

  - task: DotNetCoreCLI@2
    displayName: Publish
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/*.csproj'
      arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingFirectory) --no-restore' 
    
  # Added due to the 'dotnet ef bundle' output argument is producing to a nested directory (migrations) that does not exist...
  - task: PowerShell@2
    displayName: Create Migrations Directory
    inputs:
      targetType: 'inline'
      workingDirectory: $(Build.ArtifactStagingDirectory)
      script: |
        New-Item 'migrations' -ItemType Directory

  - task: NuGetAuthenticate@1

  - task: DotNetCoreCLI@2
    displayName: Create migration scripts
    inputs:
      command: 'custom'
      custom: 'ef'
      arguments: 'migrations bundle --force --output $(Build.ArtifactStagingDirectory)\\migrations\\efbundle.exe --verbose'

  - task: PublishBuildArtifacts@1
    displayName: Publish artifact
    inputs:
      PathtoPublish: '$(Build.ArtifactStagingDirectory)'
      ArtifactName: 'drop'
      publishLocation: 'Container'

[kevinharing]

I experience this as an issue as well.

1. Why is the restore needed in the bundling process? I do the restore already in a separate pipeline step with a custom nuget.config.
2. There seems to be no way to verify that it is using our private nuget repository during restore (--verbosity "Normal").
3. The bundle command does not accept a --no-restore or --configfile argument.

[haexyh]

I solved this also adding the NuGet Authenticate task before dispatching the migration.