Feature request: Support alpha sidecar containers in K8s 1.28+

[hessjcg]

Background

Official support for sidecar containers will be released as alpha in K8s 1.28. KEP-753

KEP-753 solves a number of pod lifecycle problems that plague the current sidecar container pattern. With built-in sidcar support, the proxy will work better than is possible today. These are the main benfits:

• K8s will guarantee that the proxy container is ready before starting the main container.
• Jobs will automatically terminate when the main container exits. There is no need to manually stop the proxy container.

This is related to #496. The sidecar container startup can be achieved in older versions of k8s by using the ordered startup of main containers and a post-start hook. See Delaying application start until sidecar is ready blog post.

Prerequisites:

This work will be started after Kubernetes 1.28 is released with alpha support for sidecar containers

Feature Description:

** This is considered an alpha feature and should only be used for experimentation and proofs of concept. It will not be a production-ready feature until after KEP-753 is GA **

To indicate that the operator should use alpha sidecar containers:

• Run on K8s 1.28
• Add the annotation cloudsql.cloud.google.com/use-alpha-sidecars=true to the AuthProxyWorkload resource

Then the operator will create the proxy container on workloads using the new sidecar feature described in KEP-753.

By default, the operator will continue to add proxy containers as it does for all older version of K8s.

The operator will always work with any supported K8s versions available in GKE.

[enocom]

Parent issue: https://github.com/kubernetes/enhancements/issues/753.