[Question] Allow EnvoyFilter in Istio addon for common scnearios

[ddx3r]

Describe scenario Hi I'm interested in using Istio mesh add-on for AKS, but this preview is not much configurable atm.

Question Do you have some public roadmap for work on this add-on when do you plan to enable envoy filters and such, cover another Istio features?

[dpaardenkooper]

Any updates about this topic? I also have customer who have interest in de envoy filters

[metaphy6]

Yes! We need EnvoyFilters to add some important headers because our security audits complain them missing, which's not good start for startups like us who go live soon. Also, we'd like to be able to update Load Balancer properties, specifically 'externalTrafficPolicy'. For the users of cloud services providers like Azure, offering an Istio that's cut off its critical features is very discouraging

[ianphil]

This is preventing us from adopting this feature of AKS also.

[shashankbarsin]

https://learn.microsoft.com/en-us/azure/aks/istio-meshconfig - allows for editing meshconfig.

ACK on externalTrafficPolicy, will discuss options to introduce it (in ARM API for serviceMeshProfile -> ingresses or just allowing edits on the ingress object in data plane API) and get back with an update.

EnvoyFilters are currently alpha in Istio upstream. With Istio addon, one of the key things we are trying to provide is official support. With EnvoyFilters (alpha) allowing for extremely vast permutations of configurations, it becomes tough to provide support if any issue arises from potential misconfiguration on such a large surface area. Having said that, @metaphy6 we are currently evaluating the possibility of opening up very few 'safe' envoy filters for common scenarios (compression, Lua filter,..). @metaphy6, @ianphil, @danielscholl, @ddx3r if there are specific list of scenarios you are looking for, please feel free to enumerate on this issue. Also, happy to jump on a call and discuss these in more detail, please let us know if you're interested in discussing.

On public roadmap, @ddx3r this repo is the roadmap and we create separate issues for each feature we want to introduce under mesh (for example, Microsoft managed mesh CA,...). For now, I'm going to update the title of this issue to 'EnvoyFilter ask for Istio addon as the discussion above is gravitating towards that on this issue. if you have wider suggestions/asks that are not covered by our roadmap right now, please feel free to suggest. Happy to take feedback and incorporate into our roadmap.

[glorious-beard]

Can you add conversion of gRPC-Web to gRPC as one of the common scenarios?

i.e.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: grpc-web-filter
  namespace: <namespace>
spec:
  workloadSelector:
    labels:
      service: <service-name>
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        filterChain:
          filter:
            name: "envoy.http_connection_manager"
            subFilter:
              name: "envoy.router"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.grpc_web

[bmargula]

This lack of EnvoyFilter support also prevents our organization from utilizing the addon currently.

[dubniczky]

+1, it would be very useful!

[Pindar]

Hi @shashankbarsin

we are currently evaluating the possibility of opening up very few 'safe' envoy filters for common scenarios (compression, Lua filter,..). @metaphy6, @ianphil, @danielscholl, @ddx3r if there are specific list of scenarios you are looking for, please feel free to enumerate on this issue.

We currently use envoy together with OPA and Styra DAS for authorization and to enable a zero trust approach. We use the following filters, which we hope will also be counted as secure in the future.

envoy.filters.http.ext_authz envoy.filters.http.router envoy.filters.http.header_to_metadata

We followed the official documentation from Styra https://docs.styra.com/das/systems/istio/create-system and https://www.openpolicyagent.org/docs/latest/envoy-introduction/

[talex-de]

+1 this would enable OPA integration for Envoy which is a highly important topic for our company!

[syeditron]

+1 We are unable to adapt Istio as service mesh as there is on OPA integration.

[Pindar]

Has Microsoft already made a decision that can be communicated? We certainly need to have clarity to move forward.

[keithmattix]

Hi @shashankbarsin

we are currently evaluating the possibility of opening up very few 'safe' envoy filters for common scenarios (compression, Lua filter,..). @metaphy6, @ianphil, @danielscholl, @ddx3r if there are specific list of scenarios you are looking for, please feel free to enumerate on this issue.

We currently use envoy together with OPA and Styra DAS for authorization and to enable a zero trust approach. We use the following filters, which we hope will also be counted as secure in the future.

envoy.filters.http.ext_authz envoy.filters.http.router envoy.filters.http.header_to_metadata

We followed the official documentation from Styra https://docs.styra.com/das/systems/istio/create-system and https://www.openpolicyagent.org/docs/latest/envoy-introduction/

I'll let @nshankar13 and @shashankbarsin comment on the decision here, but I'll just add the external authorization should be possible without EnvoyFilter: https://istio.io/latest/docs/tasks/security/authorization/authz-custom/

[nshankar13]

Hi @Pindar @talex-de @syeditron

As @keithmattix said, OPA integration should be possible for the add-on currently with the MeshConfig + Authorization Policy (a heads up that we currently have a bug with authz policy v1 validations so you will need to use v1beta1 until the fix rolls out - ETA for this fix is mid-late July). https://istio.io/v1.10/blog/2021/better-external-authz/

Would add though that as per our support policy for MeshConfig, extensionProviders / extauthz is considered "Allowed" but "Unsupported" by Azure Support - https://learn.microsoft.com/en-us/azure/aks/istio-meshconfig#allowed-supported-and-blocked-values.

For other EnvoyFilters, we will be unblocking LuaScripts, GZIP compression, and Local Rate Limiting EnvoyFilters in the near future. Will keep you posted.

[shashankbarsin]

On OPA and custom authorization - @Pindar , @talex-de, @syeditron - the addon already allows deploying your OPA instance on the cluster (https://istio.io/latest/docs/tasks/security/authorization/authz-custom/) and using MeshConfig (https://learn.microsoft.com/en-us/azure/aks/istio-meshconfig) to point to it. Wanted to provide clarity on the allowed but unsupported point made by Niranjan - the OPA instance deployed by user will be in self-managed mode - so lifecycle management (patches and minor version upgrades for OPA) and debugging for any issues with OPA - will be user’s responsibility.

Will keep this issue open, if there’s anything missing in the custom authorization flow you are looking for and not covered by above, please let us know and we can hop on a call if required to discuss.

[Pindar]

I am grateful for your assistance. I have been able to successfully implement a demo setup that utilizes the Authorization Policy in conjunction with OPA and Styra DAS. The software in use is AKS 1.29, with the Istio add-on version 1.22.
This indicates that our use case has been successfully evaluated.

[shashankbarsin]

Update: As part of 2024-08-05 release, EnvoyFilters of the types Lua, compression, and local rate limiting are now allowed (note: allowed and not support. Any issues arising from the user's Lua script itself for example are outside Microsoft support scope). These filters address the most common asks we have so far heard from Istio add-on users.

Based on earlier responses on custom authorization, it doesn't seem like we have a pending ask for EnvoyFilter for this use case as it's possible to address custom authorization using MeshConfig.

If there are any other pending use cases which require EnvoyFilter beyond the above, please let us know on this issue (will keep it open for 1 more month) and we will evaluate those.

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has not had any activity for 21 days. It will be closed if no further activity occurs within 7 days of this comment.

[microsoft-github-policy-service[bot]]

This issue will now be closed because it hasn't had any activity for 7 days after stale. ddx3r feel free to comment again on the next 7 days to reopen or open a new issue after that time if you still have a question/issue or suggestion.