custom sidecar annotation that override the image set on the controller that can be used on the workload

[caleygoff-invitae]

Is your feature request related to a problem? We find ourselves using a custom envoy image that can be set on the appmesh-controller. I could see a possibility not wanting to use the same envoy image for whatever reason across all workloads. Perhaps because of a version change or something in envoy.

Describe the solution you'd like It would be kind of cool to add an annotation here: https://github.com/aws/aws-app-mesh-controller-for-k8s/blob/1bcc239c0586c99c18636c1143a8066f94f4136a/pkg/inject/constants.go that would let us provide a custom envoy image that would override whatever is set on the controller on a per workload basis.

Describe alternatives you've considered There isn't really an alternative right now besides setting the image on the controller and forcing everyone to switch to the new api all at once.

[achevuru]

@caleygoff-invitae Sure, we will explore the possibility and will provide an update soon.

[cgchinmay]

Hi @caleygoff-invitae we do it for Virtual Gateway https://github.com/aws/aws-app-mesh-controller-for-k8s/blob/1bcc239c0586c99c18636c1143a8066f94f4136a/pkg/inject/constants.go#L41. By other workloads, do you mean you want similar annotation to skip image override for Virtual Nodes ?

[caleygoff-invitae]

@cgchinmay correct, using a custom image for the VirtualNodes would be interesting to allow so that workloads behind the VirtualNodes could continue uninterrupted. Supposing that a workload might be using a custom envoy filter that would be required for proper function. In my case I was exploring using ext_authz and using the envoyv2 stuff. While a non issue for me, it would be nice to be able to have the controller image be able to be overridden by an annotation on the VirtualNode workload.