Replace `hack/e2e/run_tests.sh` with a makefile recipe `test.integration.gke`

[mflendrich]

Is there an existing issue for this?

• [X] I have searched the existing issues

Problem Statement

The existence of hack/e2e/run-tests.sh is causing a lot of confusion:

• it is almost identical in implementation to the test.integration make target
• its name ("e2e") misguides the reader as described in https://github.com/Kong/kubernetes-ingress-controller/issues/1605#issuecomment-1208147447

Proposed Solution

• Delete hack/e2e/run-tests.sh
• Provide a way to run integration tests on GCP (that is: provision a cluster, run integration tests, tear down the cluster) from Makefile
  • This can reuse the existing make test.integration recipe.
• Ensure that the CI workflows that mention the hack/e2e/run-tests.sh script are updated to use the new make recipe. Suggested name: test.integration.gke.
• Stop using the term "e2e" when referring to "integration tests running on GKE" because the term "e2e" is reserved for a suite of tests veryfying integrations with meshes, etc.

Additional information

Originally reported by @rainest as #2751.

@rainest suggested an alternative to the test.integration.gke target: rather than making gke or kind a qualifier in the makefile recipe name, make gke|kind a Make variable to allow all tests to be parameterized with which cluster type to run on.

Acceptance Criteria

• [ ] hack/e2e/run-tests.sh has ceased to exist
• [ ] CI workflows that were running hack/e2e/run-tests.sh still work as intended
• [ ] throughout the codebase, the term "e2e tests" means that this suite of tests is executed, and not this suite of tests.

[afzal442]

/assign

[shaneutt]

Thanks @afzal442, let us know if you run into any trouble and need help, feel free to use the comments here to ask questions. :vulcan_salute:

[afzal442]

I really appreciate your any help here. As I got around this issue, I think we have to add the script of run-script.sh into makefile as below;

.PHONY: test.integration.cp
test.integration.cp:
CLUSTER_NAME="e2e-$(uuidgen)"  \
KUBERNETES_CLUSTER_NAME="${CLUSTER_NAME}" go run hack/e2e/cluster/deploy/main.go  \
GOFLAGS="-tags=integration_tests"  \
KONG_TEST_CLUSTER="${CP}:${CLUSTER_NAME}"  \
go test -parallel "${NCPU}" -timeout $(INTEGRATION_TEST_TIMEOUT) -v  \
./test/integration/...

function cleanup() {
    go run hack/e2e/cluster/cleanup/main.go ${CLUSTER_NAME}
}
trap cleanup EXIT SIGINT SIGQUIT

.PHONY: test.integration.gke
test.integration.gke:
	@$(MAKE) _test.integration.cp \
		CP="gke"

.PHONY: test.integration.kind
test.integration.kind:
	@$(MAKE) _test.integration.cp \
		CP="kind"

[shaneutt]

Hey :wave:

Sorry, it's not clear: can you help me to understand the problem you're running into that is leading you to think we need to inline the script in the Makefile?

[afzal442]

Thanks, as per the proposed soln., we need to provide a way to run integration tests on GCP from Makefile which can reuse the existing make test.integration recipe. Then we need to figure out CI workflows that mention the hack/e2e/run-tests.sh script as mentioned by @rainest. WDYT about my above approach except CI workflow [TODO]? 😄

[jrsmroz]

Hi,

Then we need to figure out CI workflows that mention the hack/e2e/run-tests.sh script as mentioned by @rainest.

I don't get this part. Why do we need that?