PSResourceGet
PSResourceGet copied to clipboard
Not working with GitLab NuGet feed
I tried to configure PowerShell Get with an GitLab Nuget Package Repository and it fails.
PS /> $parameters = @{
Name = "GitLab"
SourceLocation = "https://git.internal/api/v4/projects/916/packages/nuget/index.json"
PublishLocation = "https://git.internal/api/v4/projects/916/packages/nuget/index.json"
InstallationPolicy = 'Trusted'
Credential = $(Get-PSCredential)
}
PS /> Register-PSRepository @parameters
WARNING: Unable to resolve package source 'https://git.internal/api/v4/projects/916/packages/nuget/index.json'.
This repository works fine with nuget.exe, but not with PowerShell get.
PS /> nuget source Add -Name "GitLab" -Source "https://git.internal/api/v4/projects/916/packages/nuget/index.json" -UserName "gitlab+deploy-token-11" -Password "eobaiX8ivaich9ooh8Ei"
I'm using a gitlab deploy token in both cases (credentials changed of course).
Within powershell it doesn't work and only outputs errors:
PS /> Publish-Module -Name PSReadLine -Repository GitLab -NuGetApiKey "a" -RequiredVersion 2.0.3 -Credential $gitlabToken
Write-Error: /home/user/.local/share/powershell/Modules/PowerShellGet/2.2.4.1/PSModule.psm1:10988
Line |
10988 | … Publish-PSArtifactUtility @PublishPSArtifactUtility_Param …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Failed to publish module 'PSReadLine': 'dotnet cli failed to nuget push error: Unable to load the service index for source
| https://git.internal/api/v4/projects/916/packages/nuget/index.json. error: Response status code does not indicate success: 401 (Unauthorized). '.
GitLab is providing the NuGet Version: "3.0.0" according to the json output of the api. It doesn't use any ApiKeys.
Documentation from GitLab can be found at: https://docs.gitlab.com/ee/user/packages/nuget_repository/
Issue-Label Bot is automatically applying the label bug
to this issue, with a confidence of 0.82. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!
Links: app homepage, dashboard and code for this bot.
Any updates on this issue?
Anyone looking into this one?
@HansOMartinsen @anevjes @agowa338
I actually have this working using version 3.0.12
but it's more complicated than expected..
I believe the root of the issue is that you have to authenticate to access the feed nuget/index.json
endpoint and the Publish-PSResource
command attempts to retrieve the service index for the source without using the credential passed in.
- ~~Because you can not pass credentials to
Register-PSResourceRepository
~~ you have to configure a nuget source with stored credentials. (Even though the help for Register-PSResourceRepository says it supports them, it does not. ) - Those credentials only work for registering the source, you still have to explicitly pass credentials to the
Find-PSResource
,Install-PSResource
, andPublish-PSResource
commands.
It does not help that GitLab tokens are pretty confusing, however a deploy token is the correct one to use to publish a module from outside the pipeline. The token user names are both required and case-sensitive, however if you have the base nuget packages working it should work.
EDIT: Just noticed a couple more things.
- " Unable to load the service index for source" pretty much guarantees it is an authentication issue hitting that
nuget/index.json
endpoint. - Try using
Publish-PSResource
andRegister-PSResourceRepository
instead of the old versions. You should be passing in aSystem.Management.Automation.PSCredential
object to the-Credential
parameter for each instead of using any API key flags.
EDIT 3 :Added the version I am using, updated suspected root of the issue, and marked through some incorrect assumptions I made.
Hi @randomchance
Would you be able post your .gitlab-ci.yml
file as an example of how you got it working? Or did you get it working from outside a CI/CD pipeline?
I've been trying to get this working with a GitLab CI/CD pipeline but I haven't had much success at the moment.
Here is my .gitlab-ci.yml
file so far:
default:
image:
name: mcr.microsoft.com/powershell:7.2-alpine-3.14
stages:
- build
build-powershell:
# https://docs.gitlab.com/ee/user/packages/nuget_repository/#publish-a-nuget-package-by-using-cicd
# https://docs.gitlab.com/ee/api/packages/nuget.html
stage: build
only:
- main
- tags
- merge_requests
script:
- pwsh -Command Install-Module -Name "PowerShellGet" -AllowPrerelease -Force
# https://github.com/PowerShell/PowerShellGet/tree/master/help
- pwsh -Command Register-PSResourceRepository -Name "GitLab" -Uri "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/nuget/index.json" -Trusted -Verbose
# Custom CI variable BUILD_DEPLOY_TOKEN created via GitLab UI
# Value is set to generated deploy token with username "gitlab-deploy-token"
# Deploy token has all scopes enabled
# Several quotes to make sure the variable is expanded correctly
# When using double quotes for a command passed to "pwsh -Command" it doesn't allow "=" character
# e.g., pwsh -Command "$SecureString = ConvertTo-SecureString ..."
- echo 'ConvertTo-SecureString -String '"$BUILD_DEPLOY_TOKEN"' -AsPlainText -Force | ConvertFrom-SecureString | Set-Content -Path token' | pwsh -Command -
- pwsh -Command '$SecureString = Get-Content -Path token | ConvertTo-SecureString; $Creds = New-Object -TypeName PSCredential -ArgumentList "gitlab-deploy-token", $SecureString; Publish-PSResource -Path "$(pwd)/ModuleName" -Repository "GitLab" -Credential $Creds -Verbose'
Fails with:
Publish-PSResource: Unable to load the service index for source https://gitlab.<domain>.com/api/v4/projects/<id>/packages/nuget/index.json.
I've also tried using the CI_JOB_TOKEN
which also fails with the same error as above:
- echo 'ConvertTo-SecureString -String '"${CI_JOB_TOKEN}"' -AsPlainText -Force | ConvertFrom-SecureString | Set-Content -Path token' | pwsh -Command -
- pwsh -Command '$SecureString = Get-Content -Path token | ConvertTo-SecureString; $Creds = New-Object -TypeName PSCredential -ArgumentList "gitlab-ci-token", $SecureString; Publish-PSResource -Path "$(pwd)/ModuleName" -Repository "GitLab" -Credential $Creds -Verbose'
Hey @dbrennand, are you using a project access token or a deploy token?
It turns out that their nuget implementation only works with Personal access tokens, or deploy tokens. It's super annoying and I only found reference to that in a single sentance buried in their nuget specific "You should not need this use a nuget client instead" API docs.
You also need to have nuget present on the system. We ended up using one of the .Net SDK images, but other ways should work.
I have made a habit of pushing pwsh to all the images we use (because we have to install internal certs in all of our images anyway) so I am not 100% sure if it was on that image to start with.
This is a working Invoke-Build
task that used preview... 12? I think?
#Synopsis: Publish the built Module
Add-BuildTask Publish {
try {
$null = & dotnet nuget remove source $PublishRepoSourceName
} finally {
& dotnet nuget add source $PublishRegistry --name $PublishRepoSourceName --username $PublishUserName --password $PublishPassword '--store-password-in-clear-text'
}
if ($null -ne (Get-PSResourceRepository -Name $PublishRepoSourceName -ErrorAction Ignore)) {
Unregister-PSResourceRepository -Name $PublishRepoSourceName
}
Register-PSResourceRepository -Name $PublishRepoSourceName -URL $PublishRegistry -Trusted -Priority 1
$password = ConvertTo-SecureString $PublishPassword -AsPlainText -Force
$PublishCred = New-Object System.Management.Automation.PSCredential ($PublishUserName , $password)
Publish-PSResource -Credential $PublishCred -Repository $PublishRepoSourceName -Path $ModuleBuildPath
}
Another odd limitation we encountered was that it just fails on Windows Nano
server images, if you use a Windows image ( seems like you are not) you need to you a ServerCore
image instead.
Hey @dbrennand, are you using a project access token or a deploy token?
It turns out that their nuget implementation only works with Personal access tokens, or deploy tokens. It's super annoying and I only found reference to that in a single sentance buried in their nuget specific "You should not need this use a nuget client instead" API docs.
You also need to have nuget present on the system. We ended up using one of the .Net SDK images, but other ways should work.
I have made a habit of pushing pwsh to all the images we use (because we have to install internal certs in all of our images anyway) so I am not 100% sure if it was on that image to start with.
This is a working
Invoke-Build
task that used preview... 12? I think?#Synopsis: Publish the built Module Add-BuildTask Publish { try { $null = & dotnet nuget remove source $PublishRepoSourceName } finally { & dotnet nuget add source $PublishRegistry --name $PublishRepoSourceName --username $PublishUserName --password $PublishPassword '--store-password-in-clear-text' } if ($null -ne (Get-PSResourceRepository -Name $PublishRepoSourceName -ErrorAction Ignore)) { Unregister-PSResourceRepository -Name $PublishRepoSourceName } Register-PSResourceRepository -Name $PublishRepoSourceName -URL $PublishRegistry -Trusted -Priority 1 $password = ConvertTo-SecureString $PublishPassword -AsPlainText -Force $PublishCred = New-Object System.Management.Automation.PSCredential ($PublishUserName , $password) Publish-PSResource -Credential $PublishCred -Repository $PublishRepoSourceName -Path $ModuleBuildPath }
Another odd limitation we encountered was that it just fails on Windows
Nano
server images, if you use a Windows image ( seems like you are not) you need to you aServerCore
image instead.
@randomchance - Thank you for your reply 🙂
I did try with a deploy token but I couldn't get it working. I haven't tried using a PAT yet. I'll try that and see how I get on.
EDIT: My pipeline variable is definitely passing the deploy token correctly. So I'm not entirely sure what the issue is. The permissions scope for the deploy token is also OK with read_package_registry
, write_package_registry
@dbrennand make sure you try running nuget add source
before you run Register-PSResourceRepository
.