add featuregate for k8s 1.28 native sidecar container

[frzifus]

Description: <Describe what has changed.>

Link to tracking Issue(s): <Issue number if applicable>

• Resolves: #2376

Testing: <Describe what testing was performed and which tests were added.>

Documentation: <Describe the documentation added.>

[swiatekm]

I think this makes sense... is there a way to write an e2e for this?

Sure, you'd need to add a new test suite and run it with the gate enabled. Then checking if the sidecar is an initContainer should be straightforward.

[frzifus]

@swiatekm whats the best way to configure a k8s featuregate for a specific e2e test suite?

Do we need to create a new e.g. kind-1.30-native-sidecar.yaml and register it here? https://github.com/open-telemetry/opentelemetry-operator/blob/a515234b34557a325f0f944ac97a7f22a9b8f9fb/.github/workflows/e2e.yaml#L22-L24

Do we need then to create a new step like e2e-test-sidecar here too? https://github.com/open-telemetry/opentelemetry-operator/blob/a515234b34557a325f0f944ac97a7f22a9b8f9fb/.github/workflows/e2e.yaml#L14

[swiatekm]

@swiatekm whats the best way to configure a k8s featuregate for a specific e2e test suite?

Do we need to create a new e.g. kind-1.30-native-sidecar.yaml and register it here?

https://github.com/open-telemetry/opentelemetry-operator/blob/a515234b34557a325f0f944ac97a7f22a9b8f9fb/.github/workflows/e2e.yaml#L22-L24

Do we need then to create a new step like e2e-test-sidecar here too?

https://github.com/open-telemetry/opentelemetry-operator/blob/a515234b34557a325f0f944ac97a7f22a9b8f9fb/.github/workflows/e2e.yaml#L14

You should do the same thing we do for enabling other options in e2e tests. See here.

You do need to add a new entry to the e2e test matrix, because you need a separate operator deployment with the flag enabled. Then you add a make target that adds the command-line option to the operator Deployment (same as we do for other operator flags), and use that make target for setup in the e2e tests.

[frzifus]

Seems we dont need to enable it in Kind. https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/

Provided that your cluster has the SidecarContainers feature gate enabled (the feature is active by default since Kubernetes v1.29), you can specify a restartPolicy for containers listed in a Pod's initContainers field.

[frzifus]

Updated the changelog @pavolloffay

[frzifus]

Changed it to k8s 1.29 and we now mention in the fg desc that it should only be enabled on k8s v1.29+.