Not working with GitLab NuGet feed

[agowa]

I tried to configure PowerShell Get with an GitLab Nuget Package Repository and it fails.

PS /> $parameters = @{
  Name = "GitLab"
  SourceLocation = "https://git.internal/api/v4/projects/916/packages/nuget/index.json"
  PublishLocation = "https://git.internal/api/v4/projects/916/packages/nuget/index.json"
  InstallationPolicy = 'Trusted'
  Credential = $(Get-PSCredential)
}
PS /> Register-PSRepository @parameters
WARNING: Unable to resolve package source 'https://git.internal/api/v4/projects/916/packages/nuget/index.json'.

This repository works fine with nuget.exe, but not with PowerShell get.

PS /> nuget source Add -Name "GitLab" -Source "https://git.internal/api/v4/projects/916/packages/nuget/index.json" -UserName "gitlab+deploy-token-11" -Password "eobaiX8ivaich9ooh8Ei"

I'm using a gitlab deploy token in both cases (credentials changed of course).

Within powershell it doesn't work and only outputs errors:

PS /> Publish-Module -Name PSReadLine -Repository GitLab -NuGetApiKey "a" -RequiredVersion 2.0.3 -Credential $gitlabToken
Write-Error: /home/user/.local/share/powershell/Modules/PowerShellGet/2.2.4.1/PSModule.psm1:10988
 Line |
10988 |  …             Publish-PSArtifactUtility @PublishPSArtifactUtility_Param …
      |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | Failed to publish module 'PSReadLine': 'dotnet cli failed to nuget push error: Unable to load the service index for source
      | https://git.internal/api/v4/projects/916/packages/nuget/index.json. error:   Response status code does not indicate success: 401 (Unauthorized). '.

GitLab is providing the NuGet Version: "3.0.0" according to the json output of the api. It doesn't use any ApiKeys.

Documentation from GitLab can be found at: https://docs.gitlab.com/ee/user/packages/nuget_repository/

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.82. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[anevjes]

Any updates on this issue?

[HansOMartinsen]

Anyone looking into this one?

[randomchance]

@HansOMartinsen @anevjes @agowa338

I actually have this working using version 3.0.12 but it's more complicated than expected..

I believe the root of the issue is that you have to authenticate to access the feed nuget/index.json endpoint and the Publish-PSResource command attempts to retrieve the service index for the source without using the credential passed in.

1. ~~Because you can not pass credentials to Register-PSResourceRepository ~~ you have to configure a nuget source with stored credentials. (Even though the help for Register-PSResourceRepository says it supports them, it does not. )
2. Those credentials only work for registering the source, you still have to explicitly pass credentials to the Find-PSResource,Install-PSResource, and Publish-PSResource commands.

It does not help that GitLab tokens are pretty confusing, however a deploy token is the correct one to use to publish a module from outside the pipeline. The token user names are both required and case-sensitive, however if you have the base nuget packages working it should work.

EDIT: Just noticed a couple more things.

• " Unable to load the service index for source" pretty much guarantees it is an authentication issue hitting that nuget/index.json endpoint.
• Try using Publish-PSResource and Register-PSResourceRepository instead of the old versions. You should be passing in a System.Management.Automation.PSCredential object to the -Credential parameter for each instead of using any API key flags.

EDIT 3 :Added the version I am using, updated suspected root of the issue, and marked through some incorrect assumptions I made.

[dbrennand]

Hi @randomchance

Would you be able post your .gitlab-ci.yml file as an example of how you got it working? Or did you get it working from outside a CI/CD pipeline?

[dbrennand]

I've been trying to get this working with a GitLab CI/CD pipeline but I haven't had much success at the moment.

Here is my .gitlab-ci.yml file so far:

default:
  image:
    name: mcr.microsoft.com/powershell:7.2-alpine-3.14

stages:
  - build

build-powershell:
  # https://docs.gitlab.com/ee/user/packages/nuget_repository/#publish-a-nuget-package-by-using-cicd
  # https://docs.gitlab.com/ee/api/packages/nuget.html
  stage: build
  only:
    - main
    - tags
    - merge_requests
  script:
    - pwsh -Command Install-Module -Name "PowerShellGet" -AllowPrerelease -Force
    # https://github.com/PowerShell/PowerShellGet/tree/master/help
    - pwsh -Command Register-PSResourceRepository -Name "GitLab" -Uri "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/nuget/index.json" -Trusted -Verbose
    # Custom CI variable BUILD_DEPLOY_TOKEN created via GitLab UI
    # Value is set to generated deploy token with username "gitlab-deploy-token"
    # Deploy token has all scopes enabled
    # Several quotes to make sure the variable is expanded correctly
    # When using double quotes for a command passed to "pwsh -Command" it doesn't allow "=" character
    # e.g., pwsh -Command "$SecureString = ConvertTo-SecureString ..."
    - echo 'ConvertTo-SecureString -String '"$BUILD_DEPLOY_TOKEN"' -AsPlainText -Force | ConvertFrom-SecureString | Set-Content -Path token' | pwsh -Command -
    - pwsh -Command '$SecureString = Get-Content -Path token | ConvertTo-SecureString; $Creds = New-Object -TypeName PSCredential -ArgumentList "gitlab-deploy-token", $SecureString; Publish-PSResource -Path "$(pwd)/ModuleName" -Repository "GitLab" -Credential $Creds -Verbose'

Fails with:

Publish-PSResource: Unable to load the service index for source https://gitlab.<domain>.com/api/v4/projects/<id>/packages/nuget/index.json.

I've also tried using the CI_JOB_TOKEN which also fails with the same error as above:

 - echo 'ConvertTo-SecureString -String '"${CI_JOB_TOKEN}"' -AsPlainText -Force | ConvertFrom-SecureString | Set-Content -Path token' | pwsh -Command -
 - pwsh -Command '$SecureString = Get-Content -Path token | ConvertTo-SecureString; $Creds = New-Object -TypeName PSCredential -ArgumentList "gitlab-ci-token", $SecureString; Publish-PSResource -Path "$(pwd)/ModuleName" -Repository "GitLab" -Credential $Creds -Verbose'

[randomchance]

Hey @dbrennand, are you using a project access token or a deploy token?

It turns out that their nuget implementation only works with Personal access tokens, or deploy tokens. It's super annoying and I only found reference to that in a single sentance buried in their nuget specific "You should not need this use a nuget client instead" API docs.

You also need to have nuget present on the system. We ended up using one of the .Net SDK images, but other ways should work.

I have made a habit of pushing pwsh to all the images we use (because we have to install internal certs in all of our images anyway) so I am not 100% sure if it was on that image to start with.

This is a working Invoke-Build task that used preview... 12? I think?

#Synopsis: Publish the built Module
Add-BuildTask Publish {

    try {
        $null = & dotnet nuget remove source $PublishRepoSourceName
    } finally {
        & dotnet nuget add source $PublishRegistry --name $PublishRepoSourceName --username $PublishUserName --password $PublishPassword '--store-password-in-clear-text'
    }

    if ($null -ne (Get-PSResourceRepository -Name $PublishRepoSourceName -ErrorAction Ignore)) {
        Unregister-PSResourceRepository -Name $PublishRepoSourceName
    }
    Register-PSResourceRepository -Name $PublishRepoSourceName -URL $PublishRegistry -Trusted -Priority 1

    $password = ConvertTo-SecureString $PublishPassword -AsPlainText -Force
    $PublishCred = New-Object System.Management.Automation.PSCredential ($PublishUserName , $password)
    Publish-PSResource -Credential $PublishCred -Repository $PublishRepoSourceName -Path $ModuleBuildPath
}

Another odd limitation we encountered was that it just fails on Windows Nano server images, if you use a Windows image ( seems like you are not) you need to you a ServerCore image instead.

[dbrennand]

Hey @dbrennand, are you using a project access token or a deploy token?

It turns out that their nuget implementation only works with Personal access tokens, or deploy tokens. It's super annoying and I only found reference to that in a single sentance buried in their nuget specific "You should not need this use a nuget client instead" API docs.

You also need to have nuget present on the system. We ended up using one of the .Net SDK images, but other ways should work.

I have made a habit of pushing pwsh to all the images we use (because we have to install internal certs in all of our images anyway) so I am not 100% sure if it was on that image to start with.

This is a working Invoke-Build task that used preview... 12? I think?

#Synopsis: Publish the built Module
Add-BuildTask Publish {

    try {
        $null = & dotnet nuget remove source $PublishRepoSourceName
    } finally {
        & dotnet nuget add source $PublishRegistry --name $PublishRepoSourceName --username $PublishUserName --password $PublishPassword '--store-password-in-clear-text'
    }

    if ($null -ne (Get-PSResourceRepository -Name $PublishRepoSourceName -ErrorAction Ignore)) {
        Unregister-PSResourceRepository -Name $PublishRepoSourceName
    }
    Register-PSResourceRepository -Name $PublishRepoSourceName -URL $PublishRegistry -Trusted -Priority 1

    $password = ConvertTo-SecureString $PublishPassword -AsPlainText -Force
    $PublishCred = New-Object System.Management.Automation.PSCredential ($PublishUserName , $password)
    Publish-PSResource -Credential $PublishCred -Repository $PublishRepoSourceName -Path $ModuleBuildPath
}

Another odd limitation we encountered was that it just fails on Windows Nano server images, if you use a Windows image ( seems like you are not) you need to you a ServerCore image instead.

@randomchance - Thank you for your reply 🙂

I did try with a deploy token but I couldn't get it working. I haven't tried using a PAT yet. I'll try that and see how I get on.

EDIT: My pipeline variable is definitely passing the deploy token correctly. So I'm not entirely sure what the issue is. The permissions scope for the deploy token is also OK with read_package_registry, write_package_registry

[randomchance]

@dbrennand make sure you try running nuget add source before you run Register-PSResourceRepository.