Use ephemeral containers to execute curl and kubectl in kube2e tests

[jbohanon]

Gloo Edge Product

Open Source

Gloo Edge Version

v1.16.0

Is your feature request related to a problem? Please describe.

There are generally three moving pieces in our kube2e test framework that are external to gloo:

• executing kubectl
• executing curl
• running an upstream server

Historically these have all been handled by the testrunner located in the shared repo github.com/solo-io/k8s-utils (but only ever used by gloo edge). This introduces a sluggish experience when needing to update either the curl or kubectl tool, or if we need to update the upstream server for any reason. Additionally, it's not immediately clear that the testrunner is also the test server.

Describe the solution you'd like

These three rather disparate problems should not be solved under one bloated image. Rather, each should have a reasonable, clearly defined solution. I propose the following:

• kubectl: this tool exists on the host running any given test, and should not be needed inside a pod in the cluster. The existing tools in k8s-utils/testutils/helper use kubectl in this way, and a similar implementation can be found in the GME test utils here.
• curl: ephemeral containers can be used to run curl against the gateway proxy. We have that code already written here, with example usage in this test.
• test upstream server: we would be well-suited to mimic the GME solution for running a widely used http/s echo server with well-known endpoints and responses. In that repo, the pure-go re-implementation of httpbin, go-httpbin is used with a number of helper functions to deploy it (defined here)

Describe alternatives you've considered

No response

Additional Context

The pain which instigated this issue comes from needing to use an https upstream for one of our kube2e tests, which is not currently supported in the k8s-utils helper lib. (issue, PR).

It is also very inconvenient using our current python2 test service since it returns an HTML-ified look at the root filesystem of the pod, which can (and has in the past) change with k8s version upgrades

[jbohanon]

The team consensus at sprint planning today is to punt this discussion down to road to when GGv2 team is implementing analogous kubernetes end-to-end tests. @sam-heilbron raising this issue to your visibility to show some of the options considered here.

[github-actions[bot]]

This issue has been marked as stale because of no activity in the last 180 days. It will be closed in the next 180 days unless it is tagged "no stalebot" or other activity occurs.