Dependabot removes the indirect Go dependencies that are in use since Dec 2022

[peggy-sun-fp]

Package ecosystem

go.mod

Language version

Go 1.18

dependabot.yml content

Updated dependency

Every dependency upgrade since Dec 2022, such as upgrading golang.org/x/tools from 0.4.0 to 0.5.0

What you expected to see, versus what you actually saw

Expected: It should only upgrade golang.org/x/tools and its related indirect dependencies.

Actual: Dependabot upgrades golang.org/x/tools and removes a lot of indirect dependencies that are in use, causing CI checks to fail!

Dependabot is broken in our repository, so we need to type go mod tidy to restore the dependencies deleted by dependabot.

Native package manager behavior

When I manually upgraded with the same dependency with the command go get golang.org/x/tools, none of the above dependencies were removed. Besides, they were still existing after I typed go mod tidy.

[deivid-rodriguez]

Hei!

Can you try the dry-run.rb script against your repo to see if the problem can be reproduced using that?

Alternative, can you provide manifest files so that we can try reproducing it ourselves?

Thanks!

[jeffwidman]

This may be caused by a bug in https://github.com/dependabot/dependabot-core/pull/6361... but won't know for sure until follow @deivid-rodriguez 's suggestions.

If you want to provide manifest files, best way will be through a public GitHub repo that we can point the dry-run and similar tools at ourselves.

Also, we'll want not only the go files, but also the .dependabot.yml file to see how you've configured the go module update checker...

[peggy-sun-fp]

Hi @deivid-rodriguez @jeffwidman As our repo is private, is it okay that I put dependabot.yml, go.mod, and go.sum in a new public repo for you to test?

[deivid-rodriguez]

Absolutely!

[jeffwidman]

Yes.

Since the updater runs go mod tidy, a common solution is to include a fake.go file that "dummy imports" the direct deps so that they won't get pruned by tidy:

package fakeimports

import (
	_ "github.com/path/to/package/that/we/don't/want/tidy/to/prune"
	_ "github.com/another/package"
)

[peggy-sun-fp]

Hi @deivid-rodriguez @jeffwidman , I have created a public repository and executed your dry-run script on both repositories. However, I got a lot of error messages handled error whilst updating <dependency-url>: git_dependencies_not_reachable {:"dependency-urls"=>["github.com/deliveryhero/*"]}. I guess it is because the latter dependency is private but not sure how to solve it. The dry-run results of repositories (the original private repo deliveryhero/pd-subscription and a copied public repo peggy-sun-fp/test-dependabot) can be seen here, thanks!

[deivid-rodriguez]

Thanks for providing that!

Looking at logs, this stands out to me:

=== github.com/aws/aws-sdk-go (1.44.171)
 => checking for updates 3/38
 => latest available version is 1.44.189
 => latest allowed version is 1.44.189
 => requirements to unlock: own
 => requirements update strategy: 
 => updating github.com/aws/aws-sdk-go from 1.44.171 to 1.44.189
 => handled error whilst updating github.com/aws/aws-sdk-go: git_dependencies_not_reachable {:"dependency-urls"=>["github.com/deliveryhero/pd-fluid-go-sdk"]}

Indeed it would feel like an issue due to a dependency being private, but then, why was dependabot able to figure out it needs to upgrade it, and to which version? I know nothing about to go ecosystem, so don't have the answers, but something feels off.

[jeffwidman]

@peggy-sun-fp IIRC, you can workaround the git_dependencies_not_reachable issue by setting LOCAL_GITHUB_ACCESS_TOKEN to an access token that has access to all those repos:

https://github.com/dependabot/dependabot-core/blob/d6883339398279a0b60913cb50cfda8011f07653/bin/dry-run.rb#L14-L15

As @deivid-rodriguez points out, the dry run output looks suspect, but there's also a chance it's a bug in how dry-run parses the output, not the underlying lib... so it'd be good to get a run that isn't blocked on a permissions issue and see what error it starts hitting.

IF you're still coming up empty after that, then we may need to move this one to a support request since it does sorta smell like a legit bug here that will be easier to debug if we have access to the underlying repo and I assume this is your company's paid account so obviously can't open source it.

But first try it locally with a token that has access to all those repos and see what starts breaking.

[peggy-sun-fp]

Hi @deivid-rodriguez @jeffwidman Thanks for the tip! The dry-run script worked successfully after adding LOCAL_GITHUB_ACCESS_TOKEN. However, the results differ when creating PRs. Dry-run does not remove other dependencies while a new PR does remove some. Upon checking the repository, I found a warning saying Dependabot ran out of disk space during its update.

[jeffwidman]

Hmm... For Dry run, did you also:

• set GOPRIVATE=* to bypass the go proxy
• clear the go module cache in between runs?

Sorry I forgot to mention these ahead of time, but they can influence how bugs appear due to caching/proxy stuff... see https://github.com/dependabot/dependabot-core/issues/4536#issuecomment-1004412303 for example. And sometimes other errors start getting exposed.

The "ran out of disk space" is likely a semi-spurious error, unless it repro's consistently I'd ignore it.

[rujutaghanekar]

Is there any update on the above issue? We are also facing a similar scenario where Dependabot removes an indirect dependency but manually running go mod tidy does not.

[jakecoffman]

I made a couple of large changes this year that may have fixed it:

• https://github.com/dependabot/dependabot-core/pull/12228
• https://github.com/dependabot/dependabot-core/pull/12230

I saw there was a clean public example linked above:

• https://github.com/XTLS/Xray-core/pull/3680

I was unable to reproduce the problem using Dependabot CLI and also in a PR:

• https://github.com/dsp-testing/go-unrelated-bump/pull/9/changes

If you come across this issue and think it's still happening please link to a public PR so we can debug the issue.