dependabot-core icon indicating copy to clipboard operation
dependabot-core copied to clipboard

After latest release dependabot consistently fails with `Malformed version number string...`

Open danmoseley opened this issue 2 years ago • 31 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Package ecosystem

nuget

Package manager version

No response

Language version

No response

Manifest location and content before the Dependabot update

No response

dependabot.yml content

config https://github.com/dotnet/aspire/blob/main/.github/dependabot.yml where property is set https://github.com/dotnet/aspire/blob/c6545a2b60213d3095865b9dda60553aaef63aed/eng/Versions.props#L37 where property is consumed https://github.com/dotnet/aspire/blob/c6545a2b60213d3095865b9dda60553aaef63aed/Directory.Packages.props#L26

Updated dependency

n/a

What you expected to see, versus what you actually saw

I appear to be hitting something similar (not resolving variables) to #8470. It has broken dependabot in https://github.com/dotnet/aspire. Bug seems to be related to the changes to how property values and imports are followed in MSBuild format files.

Detail: it seems to be not correctly discovering that versions.props (link above) is getting imported into the project files along with directory.packages.props (link above). I don't see any relevant changes in our repo. So I came to look here and I see a lot of changes in the last few hours in what seems like relevant code https://github.com/dependabot/dependabot-core/commits/main/nuget/lib/dependabot/nuget and then I found this issue.

here are the github ID's if that is useful - not sure whether you can access or not. Version update 755234367 Errored with the message "Dependabot encountered an unknown error" // soon after last passing one Rebase update 754804010 passed // this was at ~7am PST or so on 11/27, the last time it passed Version update 754804004 passed

updater | 2023/11/28 03:01:48 ERROR <job_755234367> Malformed version number string $MicrosoftAspNetCoreOpenApiPackageVersion
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:223:in `initialize'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/common/lib/dependabot/version.rb:9:in `initialize'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/version.rb:27:in `initialize'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:313:in `block (2 levels) in package_versions'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:308:in `block in package_versions'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:305:in `each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:305:in `package_versions'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:296:in `find_package_version'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:286:in `dependency_requirement'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:87:in `block in parse_dependencies'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.15.4-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:85:in `parse_dependencies'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser/project_file_parser.rb:52:in `dependency_set'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser.rb:39:in `block in project_file_dependencies'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser.rb:37:in `each'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser.rb:37:in `project_file_dependencies'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/nuget/lib/dependabot/nuget/file_parser.rb:25:in `parse'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:101:in `parse_files!'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:92:in `initialize'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:24:in `new'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:24:in `create_from_job_definition'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:17:in `perform_job'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:53:in `run'
updater | 2023/11/28 03:01:48 ERROR <job_755234367> bin/update_files.rb:24:in `<main>'
updater | 2023/11/28 03:01:48 INFO <job_755234367> Sending event 3b8639905565458ea589f47c335affdb to Sentry
  proxy | 2023/11/28 03:01:48 [934] POST https://sentry.io:443/api/1451818/store/
  proxy | 2023/11/28 03:01:48 [934] 200 https://sentry.io:443/api/1451818/store/
updater | 2023/11/28 03:01:49 INFO <job_755234367> Finished job processing
updater | 2023/11/28 03:01:49 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.

@brettfo

Native package manager behavior

n/a

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

danmoseley avatar Nov 30 '23 00:11 danmoseley

Let me know if you need any more info or I can help test something for you. I see we have the same employer: my internal alias, if you want me on chat, is in my Github profile.

danmoseley avatar Nov 30 '23 00:11 danmoseley

I was hoping https://github.com/dependabot/dependabot-core/pull/8498 would fix this, but we're still failing (there's a comment there from yesterday that it was already deployed)

danmoseley avatar Dec 01 '23 18:12 danmoseley

@deivid-rodriguez do you think this is something that will likely get attention soon, or maybe not? Looks like several of us are stuck with broken dependabot. cc @austindrenski from the duplicate issue.

danmoseley avatar Jan 03 '24 21:01 danmoseley

We are also facing similar issue with our dependabot

/usr/local/lib/ruby/3.1.0/rubygems/version.rb:223:in initialize': Malformed version number string $AspNetCoreVersion (ArgumentError) from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.239.0/lib/dependabot/version.rb:9:in initialize' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/version.rb:27:in initialize' from /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in new' from /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in new' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:390:in block (2 levels) in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in block in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in upto' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:385:in block in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:382:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:382:in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:373:in find_package_version' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:363:in dependency_requirement' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:120:in block (2 levels) in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in block in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in upto' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:118:in block in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:114:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:114:in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:102:in parse_dependencies' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:58:in dependency_set' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser.rb:39:in block in project_file_dependencies' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser.rb:37:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser.rb:37:in project_file_dependencies' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.239.0/lib/dependabot/nuget/file_parser.rb:25:in parse' `

findajay avatar Jan 15 '24 08:01 findajay

Hi @danmoseley, sorry for the lack of reply. The azure devops team is looking into nuget regressions and I'm sure this will be eventually prioritized.

deivid-rodriguez avatar Jan 15 '24 12:01 deivid-rodriguez

@deivid-rodriguez any update? this was working fine before the and now has been continuously broken for https://github.com/dotnet/aspire. we have been updating manually for over two months, would like dependency updates again 😄

danmoseley avatar Feb 09 '24 19:02 danmoseley

@brettfo can you suggest/advise?

danmoseley avatar Feb 20 '24 17:02 danmoseley

@danmoseley That property can't be resolved because the updater has no navigation path to eng/Versions.props (i.e., there are no direct <Import Project.../> elements with that.) I'm familiar with the Arcade SDK at a user level (I was on the Roslyn and F# teams for some time); is the Arcade SDK responsible for pulling in that versions file? The part of the updater that's giving up happens before we ever try to do anything directly with MSBuild; we're attempting to manually crawl the .props/.targets files, but if that file requires the Arcade SDK to already be installed, then this code path won't work.

brettfo avatar Feb 20 '24 18:02 brettfo

Thanks @brettfo . @viktorhofer could you advise here perhaps?

danmoseley avatar Feb 21 '24 04:02 danmoseley

is the Arcade SDK responsible for pulling in that versions file?

Correct, the Arcade SDK imports the repository's Versions.props file: https://github.com/dotnet/arcade/blob/2b22a36ac8865066ae5790e6ce167a1ebf8398ec/src/Microsoft.DotNet.Arcade.Sdk/tools/DefaultVersions.props#L15

The Arcade SDK gets imported in these two places (i.e. for aspire):

  • https://github.com/dotnet/aspire/blob/0f74efc35e50a16733743d90aac8ff5b7bd27565/Directory.Build.props#L3
  • https://github.com/dotnet/aspire/blob/0f74efc35e50a16733743d90aac8ff5b7bd27565/Directory.Build.targets#L20

ViktorHofer avatar Feb 22 '24 11:02 ViktorHofer

@brettfo is dependabot not looking for implicitly imported directory.build.*'s? https://learn.microsoft.com/en-us/visualstudio/msbuild/customize-by-directory?view=vs-2022#directorybuildprops-and-directorybuildtargets

Or if it is, do you have an idea what the problem might be given the pointers above?

danmoseley avatar Feb 23 '24 20:02 danmoseley

Can we just skip these "malformed" versions (i.e. MicrosoftAspNetCoreOpenApiPackageVersion) and log a warning?

I don't understand why that should block the whole process just because the tool can't understand one line. It doesn't seem like an "Error, I can't do anything" situation.

eerhardt avatar Feb 23 '24 21:02 eerhardt

Yes, that would be a great near term help.

danmoseley avatar Feb 23 '24 21:02 danmoseley

@brettfo would that be possible to deploy in dependabot in the near term?

danmoseley avatar Feb 27 '24 00:02 danmoseley

@danmoseley We do support Directory.Build.props/.targets, but the problem is this line; we don't navigate through SDK includes and crossing that boundary is what's necessary to know to eventually pull in eng/Versions.props.

@eerhardt / @danmoseley Yes, we can add a check for a malformed/not-understood version. I'll have to see how far through the update process we are to know the proper way to back out.

brettfo avatar Feb 27 '24 18:02 brettfo

Thanks @brettfo for adding the workaround.

As for the long term fix -- I appreciate SDK includes are not an easy problem, but they probably a common problem too. Is it out of the question to run MSBuild proper (ie add dotnet into the dependabot container, etc...)? I'm guessing hand parsing will forever be a bit whackamole otherwise.

Incidentally, in hacking to try to work around this, I got errors like this below -- perhaps you can also tweak that message ("FileUpdater failed") to give more of a hint as to what might have happened so we can self-diagnose...

updater | Updating project [/home/dependabot/dependabot-updater/repo/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj]
updater |   Running for SDK-style project
updater |     Package [Microsoft.DotNet.RemoteExecutor] Does not exist as a dependency in [/home/dependabot/dependabot-updater/repo/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj].
updater | Update complete.
updater | Updating project [/home/dependabot/dependabot-updater/repo/tests/Aspire.Azure.Search.Documents.Tests/Aspire.Azure.Search.Documents.Tests.csproj]
updater |   Running for SDK-style project
updater |     Package [Microsoft.DotNet.RemoteExecutor] Does not exist as a dependency in [/home/dependabot/dependabot-updater/repo/tests/Aspire.Azure.Search.Documents.Tests/Aspire.Azure.Search.Documents.Tests.csproj].
updater | Update complete.
updater | 2024/02/27 22:50:36 INFO <job_793126217> [Transport] Sending envelope with items [event] bc4c0004680e4a0aa7b0976eccc741b1 to Sentry
updater | 2024/02/27 22:50:36 ERROR <job_793126217> Error processing Microsoft.DotNet.RemoteExecutor (Dependabot::DependabotError)
updater | 2024/02/27 22:50:36 ERROR <job_793126217> FileUpdater failed
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/dependency_change_builder.rb:69:in `run'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `bind_call'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `validate_call'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/dependency_change_builder.rb:42:in `create_from'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `bind_call'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `validate_call'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:114:in `create_change_for'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:72:in `block in compile_all_dependency_changes_for'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:38:in `each'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:38:in `compile_all_dependency_changes_for'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/refresh_group_update_pull_request.rb:116:in `dependency_change'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/refresh_group_update_pull_request.rb:99:in `perform'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:45:in `run'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:43:in `perform_job'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:36:in `run'
updater | 2024/02/27 22:50:36 ERROR <job_793126217> bin/update_files.rb:24:in `<main>'
  proxy | 2024/02/27 22:50:36 [350] POST https://sentry.io:443/api/1451818/envelope/
  proxy | 2024/02/27 22:50:36 [350] 200 https://sentry.io:443/api/1451818/envelope/

Edit: in fact, if you feel moved, any improvements to add info to existing failure paths like this would be potentially valuable in terms of enabling self-support. Eg "can't find file X listed in file Y" or "can't write to file Z" or "failed to parse file AA" such things. Something else that could potentially help folks is switches one can put in the dependabot yml to cause it to do things like ignore problematic files. I can open a separate issue on this kind of thing if that would be helpful.

danmoseley avatar Feb 27 '24 22:02 danmoseley

I've spent the morning digging into this and I have 2 things to report:

  1. Turns out we don't have property expansion hooked up for Directory.Packages.props, so even if we could navigate through the SDK import, we still would have blown up. I've filed #9151 to track the future work.
  2. The package versions coming back from Directory.Packages.props were always assumed to be a version, not a property like $(SomePackageVersion). I've relaxed that constraint and logged a warning when we encounter one of these. I'm working on this in PR #9153 and should have it ready soon. This should mitigate the worst cases of this.

brettfo avatar Feb 27 '24 23:02 brettfo

@danmoseley I just deployed this; can you please check if it fixes it for you?

abdulapopoola avatar Feb 28 '24 01:02 abdulapopoola

@abdulapopoola the plan is still to fix the actual issue, though, right?

If you are going to leave this issue closed, can you please cross-link another issue for us to watch for the eventual make dependabot work as well as it used to story?

austindrenski avatar Feb 28 '24 14:02 austindrenski

@brettfo and @edgarrs ; thoughts?

abdulapopoola avatar Feb 28 '24 15:02 abdulapopoola

@austindrenski I filed #9151 to track handling property expansion inside Directory.Packages.props, if that's what you're referring to. If there are other issues we can certainly link them here, but "make dependabot work as well as it used to" isn't something I can get traction on.

brettfo avatar Feb 28 '24 16:02 brettfo

They might be referring to the various bugs in the NuGet support that occurred since the changes for AzDo were merged in at the end of 2023. I've found at least four: https://github.com/dependabot/dependabot-core/issues?q=is%3Aopen+is%3Aissue+author%3Amartincostello+label%3A%22T%3A+bug+%F0%9F%90%9E%22

martincostello avatar Feb 28 '24 16:02 martincostello

If there are specific issues related to the Nuget updater that are not documented yet as GH issues, feel free to create them and we will take a look.

edgarrs avatar Feb 28 '24 16:02 edgarrs

@danmoseley I just deployed this; can you please check if it fixes it for you?

Hmm, not sure I see any difference. I just ran it in dotnet/aspire, and still get errors like

updater |   Running for SDK-style project
updater | Unhandled exception: System.IO.InvalidDataException: Property 'NetCurrent' was not found.

that's because all the csproj's start with

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>$(NetCurrent)</TargetFramework>

Where $(NetCurrent) is defined in the SDK. With your change @brettfo I expected dependabot to just pretend $(NetCurrent) evaluated to blank, and continue. It seems it's still bailing out. So I'm getting zero updates.

danmoseley avatar Feb 28 '24 16:02 danmoseley

@brettfo Thanks for making the changes but it's still failing for me.

'/usr/local/lib/ruby/3.1.0/rubygems/version.rb:223:in initialize': Malformed version number string $AspNetCoreVersion (ArgumentError) from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.245.0/lib/dependabot/version.rb:30:in initialize' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:1547:in bind_call' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:1547:in block in create_validator_procedure_medium1' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/version.rb:32:in initialize' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:1547:in bind_call' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:1547:in block in create_validator_procedure_medium1' from /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in new' from /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in new' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.245.0/lib/dependabot/version.rb:48:in new' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:968:in bind_call' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11267/lib/types/private/methods/call_validation_2_7.rb:968:in block in create_validator_method_medium1' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:411:in block (2 levels) in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in block in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in upto' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:406:in block in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:403:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:403:in package_versions' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:394:in find_package_version' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:384:in dependency_requirement' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:162:in block (2 levels) in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in block in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in upto' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/nokogiri-1.16.2-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:160:in block in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:156:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:156:in add_global_package_references' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:144:in parse_dependencies' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser/project_file_parser.rb:64:in dependency_set' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser.rb:44:in block in project_file_dependencies' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser.rb:42:in each' from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-nuget-0.245.0/lib/dependabot/nuget/file_parser.rb:42:in `project_file_dependencies' from /home/dependabot/dependabot-upda

findajay avatar Feb 28 '24 16:02 findajay

if I understand the log, it does seem like there was indeed a deployment in that these two SHA's changed since yesterday. I am guessing they indicate the dependabot version (?)

  proxy | 2024/02/28 16:55:21 proxy starting, commit: 8a64f9af83ffd9aa7eb086707544904ff53793c2
  proxy | 2024/02/28 16:55:21 * authenticating nuget feed request (host: api.nuget.org, bearer auth)
  proxy | 2024/02/28 16:55:21 Listening (:1080)
updater | 2024-02-28T16:55:21.900172856 [793513393:main:WARN:src/devices/src/legacy/serial.rs:222] Detached the serial input due to peer close/error.
updater | time="2024-02-28T16:55:23Z" level=info msg="guest starting" commit=9898ce404110b14344ddc4a580b3f18395d9e45f

danmoseley avatar Feb 28 '24 17:02 danmoseley

@brettfo @edgarrs ?

danmoseley avatar Mar 08 '24 02:03 danmoseley

@danmoseley Looking at the $(NetCurrent) issue, it's failing because there is no property <NetCurrent>net8.0</NetCurrent> (or whatever it's set to) defined in the repo. Is this a property set by Arcade? We certainly shouldn't be throwing InvalidDataException if a property can't be resolved, but since we don't yet navigate through SDK imports, it doesn't look like dependabot updates will work for this repo.

Edit: PR to not throw on an unresolved property: #9252. The issue still remains that if the updater can't determine the TFM, we don't know if a package update is possible, but we have some current work to use a real MSBuild evaluation that should allow navigating through SDK references. No ETA on that, but it is actively happening.

brettfo avatar Mar 08 '24 17:03 brettfo

@findajay Some malformed version strings are obviously still getting through, can you point me to the repo/.csproj where you're seeing this occur? I want to make sure I have the appropriate scenarios covered.

brettfo avatar Mar 08 '24 17:03 brettfo

I have a similar error - And I'd like to know if it is the same OR I should log a new error.

Suspected important part of error

proxy | 2024/04/19 13:23:42 [016] 200 https://registry.hub.docker.com:443/v2/sbtscala/scala-sbt/tags/list updater | 2024/04/19 13:23:42 ERROR <job_817148696> Error processing sbtscala/scala-sbt (ArgumentError) updater | 2024/04/19 13:23:42 ERROR <job_817148696> Malformed version number string 7_1.9.9_3.4.1

Not sure if this might be caused by some sbt scala matching code, but I was under the impression this was not working yet (as per https://github.com/dependabot/dependabot-core/issues/352)

The issue seems to be when attempting to parse the docker image sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.9.9_3.4.1

The Full error log:

updater | 2024/04/19 13:23:40 INFO <job_817148696> Starting job processing
updater | 2024/04/19 13:23:41 INFO <job_817148696> Starting update job for mdsol/sensorcloud-archon-consumer
updater | 2024/04/19 13:23:41 INFO <job_817148696> Checking all dependencies for version updates...
updater | 2024/04/19 13:23:41 INFO <job_817148696> Checking if sbtscala/scala-sbt eclipse-temurin-jammy-17.0.10_7_1.9.9_3.4.1 needs updating
  proxy | 2024/04/19 13:23:41 [012] GET https://registry.hub.docker.com:443/v2/sbtscala/scala-sbt/tags/list
  proxy | 2024/04/19 13:23:41 [012] 401 https://registry.hub.docker.com:443/v2/sbtscala/scala-sbt/tags/list
  proxy | 2024/04/19 13:23:41 [012] Remote response: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"sbtscala/scala-sbt","Action":"pull"}]}]}
  proxy | 2024/04/19 13:23:41 [014] GET https://auth.docker.io:443/token?service=registry.docker.io&scope=repository%3Asbtscala%2Fscala-sbt%3Apull&account
  proxy | 2024/04/19 13:23:41 [014] 200 https://auth.docker.io:443/token?service=registry.docker.io&scope=repository%3Asbtscala%2Fscala-sbt%3Apull&account
  proxy | 2024/04/19 13:23:41 [016] GET https://registry.hub.docker.com:443/v2/sbtscala/scala-sbt/tags/list
  proxy | 2024/04/19 13:23:42 [016] 200 https://registry.hub.docker.com:443/v2/sbtscala/scala-sbt/tags/list
updater | 2024/04/19 13:23:42 ERROR <job_817148696> Error processing sbtscala/scala-sbt (ArgumentError)
updater | 2024/04/19 13:23:42 ERROR <job_817148696> Malformed version number string 7_1.9.9_3.4.1
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:223:in `initialize'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/common/lib/dependabot/version.rb:19:in `initialize'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation_2_7.rb:1547:in `bind_call'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation_2_7.rb:1547:in `block in create_validator_procedure_medium1'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/common/lib/dependabot/version.rb:24:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation_2_7.rb:968:in `bind_call'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation_2_7.rb:968:in `block in create_validator_method_medium1'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/version.rb:29:in `initialize'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /usr/local/lib/ruby/3.1.0/rubygems/version.rb:204:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/common/lib/dependabot/version.rb:24:in `new'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `bind_call'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `validate_call'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/_methods.rb:272:in `block in _on_method_added'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:273:in `comparable_version_from'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:156:in `remove_version_downgrades'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:117:in `fetch_latest_tag'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:105:in `latest_tag_from'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:98:in `latest_version_from'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/docker/lib/dependabot/docker/update_checker.rb:19:in `latest_version'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:181:in `all_versions_ignored?'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:84:in `check_and_create_pull_request'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:64:in `check_and_create_pr_with_error_handling'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:39:in `block in perform'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:39:in `each'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:39:in `perform'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:45:in `run'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:44:in `block in perform_job'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `block in in_span'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `block in with_span'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/context.rb:87:in `with_value'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `with_span'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `in_span'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:18:in `perform_job'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:37:in `run'
updater | 2024/04/19 13:23:42 ERROR <job_817148696> bin/update_files.rb:44:in `<main>'
updater | 2024/04/19 13:23:42 INFO <job_817148696> Finished job processing
updater | 2024/04/19 13:23:42 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +------------------------------------+
updater | |   Dependencies failed to update    |
updater | +--------------------+---------------+
updater | | sbtscala/scala-sbt | unknown_error |
updater | +--------------------+---------------+
updater | time="2024-04-19T13:23:42Z" level=info msg="task complete" container_id=job-817148696-updater exit_code=0 job_id=817148696 step=updater
Footer
© 2024 GitHub, Inc.
Footer navigation
[Terms](https://docs.github.com/site-policy/github-terms/github-terms-of-service)
[Privacy](https://docs.github.com/site-policy/privacy-policies/github-privacy-statement)
[Security](https://github.com/security)
[Status](https://www.githubstatus.com/)

dependabot.yml - simple - Finds the docker file but then gets the malformed error.

version: 2

updates:
  - package-ecosystem: "docker"
    directory: "/"
    schedule:
      interval: "weekly"
    labels:
      - "dependencies"
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"
    labels:
      - "dependencies"

Docker File:

FROM sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.9.9_3.4.1 as base

#Update OS to latest
RUN apt-get update && apt-get upgrade -y

RUN mkdir -p /opt/
COPY . /opt/
WORKDIR /opt/
RUN mv build.sbt build.sbt.bck
RUN cat build.sbt.bck| sed -r 's/.*% +Test.*//' > build.sbt
RUN sbt compile
CMD sbt run

The path for the Docker should be valid as per https://hub.docker.com/r/sbtscala/scala-sbt/tags?page=&page_size=&ordering=&name=jammy-17

https://hub.docker.com/layers/sbtscala/scala-sbt/eclipse-temurin-jammy-17.0.10_7_1.9.9_3.4.1/images/sha256-3be99b57685f88870170ae32370a3ad2a448bffff739acbfcdd1ca61271b75d2?context=explore

Dependabot parsing error ?? Thanks for any ideas!

lreed-mdsol avatar Apr 19 '24 18:04 lreed-mdsol