Documentation Request: How to correctly run the test suite

[diranged]

Is this a BUG REPORT or FEATURE REQUEST?:

Documentation request

What happened:

It took a while for me to figure out how to get the test suite up and running. I don't have it perfect yet, and I wonder what requirements there really are. It would be good to clean up the developer documentation so that an outside developer can quickly get started.

How to reproduce it (as minimally and precisely as possible):

Here's the current failure I am seeing:

[diranged@ip-192-168-208-11 iam-manager ]$ KUBECONFIG=~/.kube/config KUBERNETES_SERVICE_HOST=foo KUBERNETES_SERVICE_PORT=bar LOCAL=true make test
 setting up env variables for test…
go get -u github.com/golang/mock/mockgen
go: found github.com/golang/mock/mockgen in github.com/golang/mock v1.4.4
go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1
go: golang.org/x/tools upgrade => v0.0.0-20201218024724-ae774e9781d2
go: golang.org/x/mod upgrade => v0.4.0
mockgen is in progess
/Users/diranged/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
controllers/iamrole_controller_test.go
/Users/diranged/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/diranged/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd_no_webhook/bases
go test ./... -coverprofile cover.out
?   	github.com/keikoproj/iam-manager	[no test files]
?   	github.com/keikoproj/iam-manager/api/v1alpha1	[no test files]
OK: 2 passed
Running Suite: Controller Suite
===============================
Random Seed: 1608319983
Will run 3 of 3 specs

STEP: bootstrapping test environment
2020-12-18T11:33:04.323-0800	DEBUG	controller-runtime.test-env	starting control plane	{"api server flags": []}
2020-12-18T11:33:04.333-0800	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 0, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/diranged/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:270
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:232
github.com/keikoproj/iam-manager/controllers_test.glob..func2
	/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:59
reflect.Value.call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:476
reflect.Value.Call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:337
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runAsync.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-12-18T11:33:04.336-0800	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 1, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/diranged/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:270
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:232
github.com/keikoproj/iam-manager/controllers_test.glob..func2
	/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:59
reflect.Value.call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:476
reflect.Value.Call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:337
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runAsync.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-12-18T11:33:04.340-0800	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 2, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/diranged/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:270
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:232
github.com/keikoproj/iam-manager/controllers_test.glob..func2
	/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:59
reflect.Value.call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:476
reflect.Value.Call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:337
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runAsync.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-12-18T11:33:04.343-0800	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 3, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/diranged/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:270
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:232
github.com/keikoproj/iam-manager/controllers_test.glob..func2
	/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:59
reflect.Value.call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:476
reflect.Value.Call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:337
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runAsync.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-12-18T11:33:04.346-0800	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 4, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/diranged/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:270
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/Users/diranged/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:232
github.com/keikoproj/iam-manager/controllers_test.glob..func2
	/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:59
reflect.Value.call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:476
reflect.Value.Call
	/usr/local/Cellar/go/1.15.5/libexec/src/reflect/value.go:337
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runAsync.func1
	/Users/diranged/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
Failure [0.024 seconds]
[BeforeSuite] BeforeSuite 
/Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:50

  Unexpected error:
      <*fmt.wrapError | 0xc000714c00>: {
          msg: "failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory",
          err: {
              Op: "fork/exec",
              Path: "/usr/local/kubebuilder/bin/etcd",
              Err: 0x2,
          },
      }
      failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory
  occurred

  /Users/diranged/go/src/github.com/keikoproj/iam-manager/controllers/suite_test.go:60
------------------------------

Ran 3 of 0 Specs in 0.025 seconds
FAIL! -- 0 Passed | 3 Failed | 0 Pending | 0 Skipped
--- FAIL: TestAPIs (0.02s)
FAIL
coverage: 2.3% of statements
FAIL	github.com/keikoproj/iam-manager/controllers	1.764s
ok  	github.com/keikoproj/iam-manager/internal/config	0.444s	coverage: 69.6% of statements
ok  	github.com/keikoproj/iam-manager/internal/utils	1.419s	coverage: 82.8% of statements
ok  	github.com/keikoproj/iam-manager/pkg/awsapi	1.478s	coverage: 86.0% of statements
?   	github.com/keikoproj/iam-manager/pkg/awsapi/mocks	[no test files]
?   	github.com/keikoproj/iam-manager/pkg/k8s	[no test files]
?   	github.com/keikoproj/iam-manager/pkg/log	[no test files]
ok  	github.com/keikoproj/iam-manager/pkg/validation	0.425s	coverage: 87.4% of statements
FAIL
make: *** [test] Error 1