set container of kubectl to 'uncaptured'

[silenceshell]

When trying the perf/benchmark, I met a problem below. The pods fortioclient-54894c5758-htdgl default container is captured, so the kubectl command will try to run fortio in captured container. Here is the log.

(benchmark) root@bbf4b2393670:~/tools-master/perf/benchmark# python runner/runner.py --conn 10  --qps 100,500,1000,2000,4000 --duration 240 --load_gen_type=fortio --telemetry_mode=v2-nullvm --perf=true

-------------- Running in both mode --------------
ed78b58c_qps_100_c_10_1024_v2-nullvmbothsidecars_perf.data
kubectl --namespace twopods-istio exec fortioclient-54894c5758-htdgl  -- fortio load  -jitter=False -c 10 -qps 100 -t 240s -a -r 0.001   -httpbufferkb=128 -labels ed78b58c_qps_100_c_10_1024_v2-nullvm_both http://fortioserver:8080/echo?size=1024
Defaulting container name to captured.
Use 'kubectl describe pod/fortioclient-54894c5758-htdgl -n twopods-istio' to see all of the containers in this pod.
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"fortio\": executable file not found in $PATH": unknown
command terminated with exit code 126

Maybe it is better to specify the container to run by kubectl, thus we won't exec fortio in wrong container.

[istio-policy-bot]

😊 Welcome @silenceshell! This is either your first contribution to the Istio tools repo, or it's been awhile since you've been here.

You can learn more about the Istio working groups, code of conduct, and contributing guidelines by referring to Contributing to Istio.

Thanks for contributing!

Courtesy of your friendly welcome wagon.

[silenceshell]

/retest

[silenceshell]

I think what we actually want is to control this with kubectl.kubernetes.io/default-container

https://kubernetes.io/docs/reference/labels-annotations-taints/#kubectl-kubernetes-io-default-container

This is a good choice, my only concern is backward compatible, as default-container-annotation is introduced to k8s from v1.21, this may result in istio benchmark tools not working on older k8s clusters.

[astronaut0131]

-------------- Running in both mode --------------
e6f01502_qps_1000_c_2_1024_v2-stats-nullvmbothsidecars_perf.data
kubectl --namespace twopods-istio exec fortioclient-69bd59f79b-xkqn9  -- fortio load  -jitter=True -c 2 -qps 1000 -t 240s -a -r 0.001   -httpbufferkb=128 -labels e6f01502_qps_1000_c_2_1024_v2-stats-nullvm_both http://fortioserver:8080/echo?size=1024
OCI runtime exec failed: exec failed: unable to start container process: exec: "fortio": executable file not found in $PATH: unknown

Same problem here, runner.py can not even work correctly, why this PR is blocking?