dependabot-azure-devops
dependabot-azure-devops copied to clipboard
Test for Nuget auth fix
Pushed a new image. To test:
- task: dependabot@1
inputs:
dockerImageTag: '1.25.2-pullrequest0927-0005'
// your other inputs here ...
Hi,
I tested dockerImageTag: '1.25.2-pullrequest0927-0005'
.
Not running well.
- "Parsing dependencies information" takes 5 minutes alone. Under 1.24 the whole dependabot process takes 32s for the same repo.
- Also then stuck forever (~20m) at
Finding updated dependencies for AutoMapper.Collection.
🌍 --> GET https://api.nuget.org/v3-flatcontainer/automapper.collection/9.0.0/automapper.collection.nuspec
🌍 <-- 200 https://api.nuget.org/v3-flatcontainer/automapper.collection/9.0.0/automapper.collection.nuspec
🌍 --> GET https://api.nuget.org/v3-flatcontainer/automapper/12.0.0/automapper.nuspec
🌍 <-- 200 https://api.nuget.org/v3-flatcontainer/automapper/12.0.0/automapper.nuspec
Updating AutoMapper.Collection from 7.0.1 to
running NuGet updater:
/opt/nuget/NuGetUpdater/NuGetUpdater.Cli update --repo-root /home/dependabot/dependabot-updater/tmp/<devops-collection>/<devops-project>/_git/PM.DCP.API.Service --solution-or-project /home/dependabot/dependabot-updater/tmp/<devops-collection>/<devops-project>/_git/PM.DCP.API.Service/src/PM.DCP.Service.Application/PM.DCP.Service.Application.csproj --dependency AutoMapper.Collection --new-version 9.0.0 --previous-version 7.0.1 --verbose
then eventually
[...]
/tmp/package-dependency-resolution_i3u82p/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/<devops-collection>/_packaging/P-OS-Artifacts/nuget/v3/index.json.
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/<devops-collection>/_packaging/P-OS-Artifacts/nuget/v3/index.json. [/tmp/package-dependency-resolution_i3u82p/Project.csproj]
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Response status code does not indicate success: 401 (Unauthorized). [/tmp/package-dependency-resolution_i3u82p/Project.csproj]
[...]
STDERR:
Package [AutoMapper.Collection] Does not exist as a dependency in [/home/dependabot/dependabot-updater/tmp/<devops-collection>/<devops-project>/_git/PM.DCP.API.Service/src/PM.DCP.Service.Application/PM.DCP.Service.Application.csproj].
The strange thing here is, that the package is not contained in the private feed at all. We don't use any upstream but load public packages always directly from public source (nuget.org). Only our own packages are available from the private feed. So I totally don't get what it is doing at all... Package AutoMapper.Collection needs to be loaded from nuget.org, not the private feed at all.
Will give it some tries over the weekend.
I just tried but no luck for me neither
/usr/local/lib/ruby/3.1.0/openssl/buffering.rb:214:in `sysread_nonblock': SSL_read: unexpected eof while reading (OpenSSL::SSL::SSLError) (Excon::Error::Socket)
from /usr/local/lib/ruby/3.1.0/openssl/buffering.rb:214:in `read_nonblock'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/socket.rb:209:in `read_nonblock'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/socket.rb:79:in `block in readline'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/socket.rb:70:in `loop'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/socket.rb:70:in `readline'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/response.rb:73:in `block in parse'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/response.rb:72:in `loop'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/response.rb:72:in `parse'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/connection.rb:460:in `response'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/excon-0.104.0/lib/excon/connection.rb:291:in `request'
so 'same` as here https://github.com/tinglesoftware/dependabot-azure-devops/issues/921#issuecomment-1878765220
Gave it a test run a little while ago and my result is basically the same, that it doesn't work with private registries (Azure Artifacts in my case)
Also... I tried comparing the logs between 1.24
and 1.25.2-pullrequest0927-0005
and noticed that it doesn't even check the private feed (seemingly) on the newer version.
1.24:
Checking if {packageName} 1.0.2 needs updating
🌍 --> GET https://pkgs.dev.azure.com/{organisationName}/{projectName}/_packaging/{feedName}/nuget/v3/index.json
🌍 <-- 200 https://pkgs.dev.azure.com/{organisationName}/{projectName}/_packaging/{feedName}/nuget/v3/index.json
🌍 --> GET https://pkgs.dev.azure.com/{organisationName}/2110eeea-1457-4552-8bbd-5d9d2a0a4338/_packaging/b051a3cf-2486-4e06-8810-1e4e6beabb2b/nuget/v3/flat2/{packageName}/index.json
🌍 <-- 200 https://pkgs.dev.azure.com/{organisationName}/2110eeea-1457-4552-8bbd-5d9d2a0a4338/_packaging/b051a3cf-2486-4e06-8810-1e4e6beabb2b/nuget/v3/flat2/{packageName}/index.json
🌍 --> GET https://azuresearch-usnc.nuget.org/query?q={packageName}&prerelease=true&semVerLevel=2.0.0
🌍 <-- 200 https://azuresearch-usnc.nuget.org/query?q={packageName}&prerelease=true&semVerLevel=2.0.0
Requirements to unlock own
Requirements update strategy
Updating {packageName} from 1.0.2 to 1.2.0
Submitting {packageName} pull request for creation.
1.25.2-pullrequest0927-0005:
Checking if {packageName} 1.0.2 needs updating
🌍 --> GET https://api.nuget.org/v3/registration5-gz-semver2/{packageName}/index.json
🌍 <-- 404 https://api.nuget.org/v3/registration5-gz-semver2/{packageName}/index.json
🌍 --> GET https://api.nuget.org/v3-flatcontainer/{packageName}/1.0.2/{packageName}.nuspec
🌍 <-- 404 https://api.nuget.org/v3-flatcontainer/{packageName}/1.0.2/{packageName}.nuspec
Requirements to unlock all
Requirements update strategy
Finding updated dependencies for {packageName}.
🌍 --> GET https://api.nuget.org/v3-flatcontainer/{packageName}/1.2.0/{packageName}.nuspec
🌍 <-- 404 https://api.nuget.org/v3-flatcontainer/{packageName}/1.2.0/{packageName}.nuspec
Updating {packageName} from 1.0.2 to
running NuGet updater:
/opt/nuget/NuGetUpdater/NuGetUpdater.Cli update --repo-root /home/dependabot/dependabot-updater/tmp/{organisationName}/DemoProject-Internal/_git/dependabot-github-919 --solution-or-project /home/dependabot/dependabot-updater/tmp/{organisationName}/DemoProject-Internal/_git/dependabot-github-919/source/DependabotRepro.ClassLibrary/DependabotRepro.ClassLibrary.csproj --dependency {packageName} --new-version 1.2.0 --previous-version 1.0.2 --verbose
No global.json files found.
No dotnet-tools.json files found.
Running for project [/home/dependabot/dependabot-updater/tmp/{organisationName}/DemoProject-Internal/_git/dependabot-github-919/source/DependabotRepro.ClassLibrary/DependabotRepro.ClassLibrary.csproj]
Running for SDK-style project
dotnet build in GetAllPackageDependenciesAsync failed. STDOUT: MSBuild version 17.8.3+195e7f5a3 for .NET
Determining projects to restore...
/tmp/package-dependency-resolution_CSEqxR/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{organisationName}/{projectName}/_packaging/{feedName}/nuget/v3/index.json.
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/{organisationName}/{projectName}/_packaging/{feedName}/nuget/v3/index.json. [/tmp/package-dependency-resolution_CSEqxR/Project.csproj]
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Response status code does not indicate success: 401 (Unauthorized). [/tmp/package-dependency-resolution_CSEqxR/Project.csproj]
Build FAILED.
and shortly after failing with the following stacktrace:
/home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/_types.rb:222:in `must': Passed `nil` into T.must (TypeError)
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-1bb15b002f29/common/lib/dependabot/pull_request_creator/branch_namer/solo_strategy.rb:41:in `prefixes'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-1bb15b002f29/common/lib/dependabot/pull_request_creator/branch_namer/solo_strategy.rb:32:in `new_branch_name'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-1bb15b002f29/common/lib/dependabot/pull_request_creator/branch_namer.rb:30:in `new_branch_name'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-1bb15b002f29/common/lib/dependabot/pull_request_creator.rb:305:in `azure_creator'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/call_validation.rb:272:in `bind_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/call_validation.rb:272:in `validate_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-1bb15b002f29/common/lib/dependabot/pull_request_creator.rb:235:in `create'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/call_validation.rb:272:in `bind_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/call_validation.rb:272:in `validate_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11188/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
from bin/update_script.rb:823:in `block in <main>'
from bin/update_script.rb:539:in `each'
from bin/update_script.rb:539:in `<main>'
Thanks @cyberblast @davesmits , @JensSchadron for reporting. I am also testing this and getting the same results. I will keep updating to the latest commit from the code repo to see if there are improvements about once a week.
New tag version is 1.25.2-pullrequest0927-0011
- task: dependabot@1
inputs:
dockerImageTag: '1.25.2-pullrequest0927-0011'
// your other inputs here ...
Still seeing the same issue with this one but it may behave differently on another setup.
Reports that the latest bits are working with private feeds but I still can't get it to work. Maybe someone else can?
- task: dependabot@1
inputs:
dockerImageTag: '1.25.4-pullrequest0927-0002'
// your other inputs here ...
The SSL error is at least gone.
dotnet build in GetAllPackageDependenciesAsync failed. STDOUT: MSBuild version 17.8.3+195e7f5a3 for .NET
Determining projects to restore...
/tmp/package-dependency-resolution_wbmdXd/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/pandoraintelligence/_packaging/Pandora.Platform/nuget/v3/index.json.
/tmp/package-dependency-resolution_wbmdXd/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/pandoraintelligence/_packaging/Pandora.Platform/nuget/v3/index.json.
/tmp/package-dependency-resolution_wbmdXd/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/pandoraintelligence/_packaging/Pandora.Platform/nuget/v3/index.json.
/tmp/package-dependency-resolution_wbmdXd/Project.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/pandoraintelligence/_packaging/Pandora.Platform/nuget/v3/index.json.
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/pandoraintelligence/_packaging/Pandora.Platform/nuget/v3/index.json. [/tmp/package-dependency-resolution_wbmdXd/Project.csproj]
/usr/local/dotnet/current/sdk/8.0.100/NuGet.targets(156,5): error : Response status code does not indicate success: 401 (Unauthorized). [/tmp/package-dependency-resolution_wbmdXd/Project.csproj]
I gave it a quick shot but seems to behave the same as before :(
Some more fixes have been put in. The new version for testing is 1.25.4-pullrequest0927-0011
. Still having issues with private feeds. Maybe someone else will have better luck.
- task: dependabot@1
inputs:
dockerImageTag: '1.25.4-pullrequest0927-0011'
// your other inputs here ...
As of today:
- task: dependabot@1
inputs:
dockerImageTag: '1.26.1-pullrequest0927-0038'
// your other inputs (if any) go here ...
Hey. I gave it a try (1.26.1-pullrequest0927-0038) but get some errors...
running NuGet updater:
/opt/nuget/NuGetUpdater/NuGetUpdater.Cli update --repo-root /home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo> --solution-or-project /home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/tests/<some-proj>/<some-proj>.csproj --dependency <some-private-package> --new-version 4.2.0 --previous-version 3.4.0 --verbose
No dotnet-tools.json files found.
Running for project [/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/tests/<some-proj>/<some-proj>.csproj]
Running for SDK-style project
Package [<some-private-package>] Does not exist as a dependency in [/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/tests/<some-proj>/<some-proj>.csproj].
Update complete.
/home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/configuration.rb:296:in `call_validation_error_handler_default': Parameter 'milestone': Expected type T.nilable(T.any(Integer, T::Array[String])), got type String with value "44711" (TypeError)
Caller: /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/call_validation.rb:215
Definition: /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-df3e6e1cc9f7/common/lib/dependabot/pull_request_creator.rb:174
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/configuration.rb:303:in `call_validation_error_handler'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/call_validation.rb:300:in `report_error'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/call_validation.rb:218:in `block in validate_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/signature.rb:235:in `block in each_args_value_type'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/signature.rb:229:in `each'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/signature.rb:229:in `each_args_value_type'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/call_validation.rb:215:in `validate_call'
from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11226/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
from bin/update_script.rb:801:in `new'
from bin/update_script.rb:801:in `block in <main>'
from bin/update_script.rb:540:in `each'
from bin/update_script.rb:540:in `<main>'
##[error]The process '/usr/bin/docker' failed with exit code 1
running NuGet updater:
/opt/nuget/NuGetUpdater/NuGetUpdater.Cli update --repo-root /home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo> --solution-or-project /home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/test/<some-proj>/<some-proj>.csproj --dependency Microsoft.Azure.WebJobs.Extensions.EventHubs --new-version 6.0.2 --previous-version 5.4.0 --verbose
No dotnet-tools.json files found.
Running for project [/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/test/<some-proj>/<some-proj>.csproj]
Running for SDK-style project
Package [Microsoft.Azure.WebJobs.Extensions.EventHubs] Does not exist as a dependency in [/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/home/dependabot/dependabot-updater/tmp/<some-path>/_git/<some-repo>/test/<some-proj>/<some-proj>.csproj].
Update complete.
bin/update_script.rb:747:in `block (2 levels) in <main>': undefined method `directory' for nil:NilClass (NoMethodError)
next if updated_files.first.directory != "/" && !title.end_with?(" in #{updated_files.first.directory}")
^^^^^^^^^^
from bin/update_script.rb:724:in `each'
from bin/update_script.rb:724:in `block in <main>'
from bin/update_script.rb:540:in `each'
from bin/update_script.rb:540:in `<main>'
##[error]The process '/usr/bin/docker' failed with exit code 1
I also recognized, that it is downloading the whole https://api.nuget.org/v3-flatcontainer nuspecs for every dependabot task again and again :/ we are running ~ 90 dependabot tasks in our nightly pipe (one per repo)
@cyberblast thanks for trying this one out. Unfortunately, the changes in the upstream repo are still ongoing. For the jobs you are running at night, I recommend using 1.24
until we can tell what is going on. I haven't had much time to look through the code to find a solution so hopefully it gets fixed in the upstream or a community fix.
As of today:
- task: dependabot@1
inputs:
dockerImageTag: '1.26.1-pullrequest0927-0043'
// your other inputs (if any) go here ...
As of today:
- task: dependabot@1
inputs:
dockerImageTag: '1.26.4-pullrequest0927-0002'
// your other inputs (if any) go here ...
Updated to the mentioned version, but the issue persists.
I did the try 1.26.1-pullrequest0927-0043
version and seems the SSL errors / Http Errors we had before are gone
now got this stacktrace (just lack of ruby knowledge to do something with it)
Package [Polly] Does not exist as a dependency in [/home/dependabot/dependabot-updater/tmp/pandoraintelligence/Pandora\%20Box/_git/Pandora.NuGet/home/dependabot/dependabot-updater/tmp/pandoraintelligence/Pandora\%20Box/_git/Pandora.NuGet/Pandora.Nats.IntegrationTests/Pandora.Nats.IntegrationTests.csproj].
Update complete.
bin/update_script.rb:756:in `block (2 levels) in <main>': undefined method `directory' for nil:NilClass (NoMethodError)
next if updated_files.first.directory != "/" && !title.end_with?(" in #{updated_files.first.directory}")
^^^^^^^^^^
from bin/update_script.rb:733:in `each'
from bin/update_script.rb:733:in `block in <main>'
from bin/update_script.rb:549:in `each'
from bin/update_script.rb:549:in `<main>'
As of today:
- task: dependabot@1
inputs:
dockerImageTag: '1.27.2-pullrequest0927-0002'
// your other inputs (if any) go here ...
This may still not work as we wait for https://github.com/dependabot/dependabot-core/pull/8927
As of today:
- task: dependabot@1
inputs:
dockerImageTag: '1.27.4-pullrequest0927-0002'
// your other inputs (if any) go here ...
Worked for me
oke found new minor issue; workitem linking is now broken
Did the latest changes from dependabot-core #1090 make improvements to fixing the issue?
Updated to the mentioned version, but the issue persists.
Using latest image is easier.
@mburumaxwell what do you mean with using latest image is easier? Seeing the branch is deleted not merged?
I kept this branch/PR to test changes on dependabot-core's main that had not been released/tagged without merging. All of those changes have been released/tagged recently. So using dockerImageTag: latest
right away should give you the changes on this repository's main branch. This is until I release a new version in the repository.
Thanks! I am going to rmeove the 1.24 and upgrade again 💖
Please note that you may still want to continue using dockerImageTag: '1.24'
if you were doing so until there is a release in this repo. However, if you were using image tags from this PR/branch such as dockerImageTag: '1.27.4-pullrequest0927-0002'
. You can transition to dockerImageTag: latest