Install setup-envtest module version which matching with the controller-runtime version instead of latest

[camilamacedo86]

trafficstars

What do you want to happen?

Motivation

The concern/motivation to raise this one is that in Kubebuilder, we are: (here)

ENVTEST = $(shell pwd)/bin/setup-envtest
.PHONY: envtest
envtest: ## Download envtest-setup locally if necessary.
	$(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest)

Then, a KB user scaffolds the project today. The scaffold is done with controller-runtime version X. Afterwards, we are still producing new releases and versions. Thus, this target can begin to download a version of this module that is no longer compatible.

Proposed solution

• Scaffold the package tools with the module import:

//go:build tools
// +build tools

/*
YOUR_COPYRIGHT
*/

// This package imports things required by build scripts, to force `go mod` to see them as dependencies
package tools

import (
	_ "sigs.k8s.io/controller-runtime/tools/setup-envtest"
)

• Change the makefile target to go install the setup-envtest mod version which matches with the controller-runtime version in the go.mod, e.g.:

ENVTEST = $(shell pwd)/bin/setup-envtest
.PHONY: envtest
envtest: ## Download envtest-setup locally if necessary.
	$(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest)

Then, the command go install sigs.k8s.io/controller-runtime/tools/setup-envtest ought to install its version which
corresponding with the C+R version used into the project. More info: https://github.com/kubernetes-sigs/controller-runtime/issues/1670

Extra Labels

No response

[jmrodri]

/label triage/accepted

[k8s-ci-robot]

@jmrodri: The label(s) /label triage/accepted cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor

In response to this:

/label triage/accepted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[NikhilSharmaWe]

@camilamacedo86 I would like to work on this. /assign

[camilamacedo86]

Adding here what was discussed in https://github.com/kubernetes-sigs/kubebuilder/pull/2486 for :

From @joelanford :

This setup is going to result in the project's CI dependencies being added to the main project's go.mod file, which has implications on: how underlying dependencies get resolved when a build tool and the main project share a transitive dependency (e.g. the build tool might unnecessarily hold back the transitive dependency on a lower version due to Go's MVS algorithm) how other projects consume this project as a dependency (external projects will end up with this project's build tool dependencies in their go.mod files and might impact their transitive dependency selection also due to MVS) TL;DR: At a minimum, I think we need to put the tools dependencies in a separate go module. However see also this issue for background about why even that might not be a great idea: https://github.com/kubernetes-sigs/cluster-api/issues/5569

From @camilamacedo86 :

Note that the main reason/motivation for them do not add the dep as an import in the issue shared is Also this helps to make the version of our dependencies visible top-level (in the Makefile instead of hack/tools/go.mod) which does not seems our case; we are only trying to use this approach with the env-test to ensure that its version is compatible with the controller-runtime one scaffold in the project and not all scenarios. The approach was the solution provided by the controller-runtime maintainers, see: https://github.com/kubernetes-sigs/kubebuilder/issues/2480

However, what would be your suggestion for us do not to scaffold to use env-test from the latest by default in this case?

@From @joelanford :

we are only trying to use this approach with the env-test

That may be, but many users will see this tools pattern and start adding other build dependencies, thus further polluting their main go.mod file. One of the main goals of kubebuilder is to help reinforce best practices, and I'd consider it a bad practice to include build deps alongside library deps in the same go.mod/go.sum.

One way to solve this problem is something like:

CR_VERSION=$(shell go list -m sigs.k8s.io/controller-runtime | cut -d" " -f2)
ENVTEST = $(shell pwd)/bin/setup-envtest
.PHONY: envtest
envtest: ## Download envtest-setup locally if necessary.
	$(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@$(CR_VERSION))

From @camilamacedo86 :

That does not work, see (we cannot use the tag to download the envtest, see: https://github.com/kubernetes-sigs/controller-runtime/issues/1670 ) :

 go get sigs.k8s.io/controller-runtime/tools/[email protected]
go: downloading sigs.k8s.io/controller-runtime v0.11.1
go get: module sigs.k8s.io/[email protected] found, but does not contain package sigs.k8s.io/controller-runtime/tools/setup-envtest

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[camilamacedo86]

/remove-lifecycle stale

[rumstead]

Is there a recommendation on how to get the dependencies?

sigs.k8s.io/controller-runtime/tools/setup-envtest: module sigs.k8s.io/controller-runtime@latest found (v0.12.3), but does not contain package sigs.k8s.io/controller-runtime/tools/setup-envtest

EDIT: Don't love it but it works..

.PHONY: envtest
envtest: $(ENVTEST) ## Download envtest-setup locally if necessary.
$(ENVTEST): $(LOCALBIN)
	# https://github.com/kubernetes-sigs/kubebuilder/issues/2480
	test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/[email protected]

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jroper]

The dependency on latest has just caused our build to suddenly fail without us changing anything:

22:45:19 > Downloading sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20231008020014-968daa8964f9
go: downloading go.uber.org/zap v1.26.0
go: downloading github.com/go-logr/zapr v1.2.4
go: downloading github.com/go-logr/logr v1.2.4
go: downloading golang.org/x/text v0.12.0
go: downloading go.uber.org/multierr v1.10.0
# go.uber.org/multierr
/home/circleci/go/pkg/mod/go.uber.org/[email protected]/error.go:224:20: undefined: atomic.Bool
note: module requires Go 1.19
make[1]: *** [Makefile:171: envtest] Error 2
make: *** [Makefile:114: test] Error 2

This violates the principle of repeatable builds. We shouldn't be forced to upgrade to Go 1.19 before we are ready. There needs to be a better way.

[yangsoon]

The dependency on latest has just caused our build to suddenly fail without us changing anything:

22:45:19 > Downloading sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20231008020014-968daa8964f9
go: downloading go.uber.org/zap v1.26.0
go: downloading github.com/go-logr/zapr v1.2.4
go: downloading github.com/go-logr/logr v1.2.4
go: downloading golang.org/x/text v0.12.0
go: downloading go.uber.org/multierr v1.10.0
# go.uber.org/multierr
/home/circleci/go/pkg/mod/go.uber.org/[email protected]/error.go:224:20: undefined: atomic.Bool
note: module requires Go 1.19
make[1]: *** [Makefile:171: envtest] Error 2
make: *** [Makefile:114: test] Error 2

This violates the principle of repeatable builds. We shouldn't be forced to upgrade to Go 1.19 before we are ready. There needs to be a better way.

I also encountered this issue just now. You can temporarily fix it by using the command

go install sigs.k8s.io/controller-runtime/tools/[email protected]

[Huang-Wei]

Use @latest leads to downloading deps that may need higher version of Go, which breaks my CI:

     go: downloading go.uber.org/multierr v1.10.0
     # go.uber.org/multierr
     /go/pkg/mod/go.uber.org/[email protected]/error.go:224:20: undefined: atomic.Bool
     note: module requires Go 1.19

As @jroper mentioned above, it violates the principle of repeatable builds.

[zeeke]

The dependency on latest has just caused our build to suddenly fail without us changing anything:

22:45:19 > Downloading sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20231008020014-968daa8964f9
go: downloading go.uber.org/zap v1.26.0
go: downloading github.com/go-logr/zapr v1.2.4
go: downloading github.com/go-logr/logr v1.2.4
go: downloading golang.org/x/text v0.12.0
go: downloading go.uber.org/multierr v1.10.0
# go.uber.org/multierr
/home/circleci/go/pkg/mod/go.uber.org/[email protected]/error.go:224:20: undefined: atomic.Bool
note: module requires Go 1.19
make[1]: *** [Makefile:171: envtest] Error 2
make: *** [Makefile:114: test] Error 2

This violates the principle of repeatable builds. We shouldn't be forced to upgrade to Go 1.19 before we are ready. There needs to be a better way.

I also encountered this issue just now. You can temporarily fix it by using the command

go install sigs.k8s.io/controller-runtime/tools/[email protected]

As an alternative, not sure if it's better than referencing a specific commit, you can target a branch:

 go install sigs.k8s.io/controller-runtime/tools/[email protected]
go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20231026160510-6ffcf19a340d
go: downloading sigs.k8s.io/controller-runtime v0.13.2
go: downloading golang.org/x/text v0.3.6