Switch e2e tests to a binary

[cmwaters]

Summary

Currently the knuu e2e tests are part of the go test suite. The problem is that these e2e tests are permissioned. You need to have the kube config setup to run them. We also want them separated from our unit tests and integration tests as they are part of a separate CI workflow. This problem becomes more apparent when we use knuu for performance based tests that we don't want continually running. Since environment variables are set for the lifetime of the terminal, I frequently accidentally run e2e tests when I run make test-short on my branch.

Proposal

I would like to propose that we put all our e2e tests in their own binary. To run them, one would use go run tests/e2e. We would not need to set environment variables

[rootulp]

Works for me. Can we also modify the existing make test-e2e command to run that separate binary?

[cmwaters]

As additional context. I would want the correctness tests run separately from the performance/experimental tests. These should be split into two packages like

go run ./test/e2e // for our correctness based tests i.e. compatibility and upgrades go run ./test/benchmark // for our standardised benchmarks go run ./test/experimental/two_node_test // for experimental work

(maybe the last two should be the same for now)

[ninabarbakadze]

todo: also disable throughput tests and move them to benchmark or experimental