feat: support dynamic modules

[Xunzhuo]

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes https://github.com/envoyproxy/gateway/issues/5668

Release Notes: Yes

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 78.16092% with 38 lines in your changes missing coverage. Please review.

Project coverage is 70.46%. Comparing base (96c06be) to head (5faec1c). Report is 224 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/xds/translator/dynamicmodule.go	71.02%	20 Missing and 11 partials :warning:
internal/gatewayapi/envoyextensionpolicy.go	84.00%	3 Missing and 1 partial :warning:
internal/gatewayapi/dynamicmodule.go	92.50%	2 Missing and 1 partial :warning:

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #5669    +/-   ##
========================================
  Coverage   70.45%   70.46%            
========================================
  Files         217      219     +2     
  Lines       36016    36177   +161     
========================================
+ Hits        25376    25493   +117     
- Misses       9135     9164    +29     
- Partials     1505     1520    +15     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[arkodg]

1. EEP is probably not the best place for this, since a bad module can take out the entire proxy, not just the route
2. Can you share a workflow of how a user will fetch a module into the pod

[zhaohuabing]

Some options that I can think of:

• use a docker image to download dynamic modules. This docker image can be deployed as a side car in the Envoy pod and the lib can be loaded to Envoy via a shared volume.
• use a customized envoy docker image provided by users. The libs for dynamic modules are already loaded to that image.

[Xunzhuo]

The first approach can solve the same issues for golang filter, so I'm +1 for this.

Second one needs some prerequisites for features like golang/dynamic module, use envoyproxy to customize the image before using EEP for dm/go filter

[arkodg]

Some options that I can think of:

• use a docker image to download dynamic modules. This docker image can be deployed as a side car in the Envoy pod and the lib can be loaded to Envoy via a shared volume.
• use a customized envoy docker image provided by users. The libs for dynamic modules are already loaded to that image.

this experience is not seamless, so maybe lets first document this in docs post v1.4, and then get feedback, and then we can eliminate the need for the EnvoyPatchPolicy with an API field (maybe in EnvoyProxy) in the future

[mathetake]

FYI: Dynamic modules require the exact version match (at least for now), so for example, the modules working with Envoy v1.50.0 are not guaranteed to work with v1.51.0 (if the compatibility check fails, then HTTP filter configuration xds request will be rejected). So, coupling Envoy images would be the safest way; either using the Envoy container itself or with init container downloading (with some version argument). I believe the same restriction applies to golang filter.

So either way, the life cycle of downloading into Envoy container should match the one of Envoy container itself. that's my 2p

[zhaohuabing]

@mathetake Thanks for clarifying how dynamic modules need to match the version of Enovy Proxy.

This would be a problem for upgrade - since the ABI version is defined in the abi_version.h, EG can't check if a dynamic module will still be compatible or not after upgrade. Users can only find out after something is broken.

Is there a way for EG to detect an incompatibility between Envoy and a dynamic module, and then surfaced that issue in the status of EG resources?

[mathetake]

There's no mechanism to check that without actually loading Envoy. Like i suggested, the current best way is to tie the Envoy container with modules as in the examples or EG introduce some module management init container like the container receives the Envoy version and downloads the specific dynamic modules for that version before Envoy running. To be clear, the same problem exists for golang filter already, and nothing specific to dynamic modules.

[mathetake]

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

[arkodg]

yah above approach looks good @mathetake, this is maybe hard to predict, but any guesstimate on average and max sizes of the module ?

[mathetake]

but any guesstimate on average and max sizes of the module ?

shared library is almost the same as normal application binary, so the question would be how large a normal application would be; a few MB at least for Rust and Go, and tens of MB if it contains debug info:

~/dynamic-modules-examples/rust$ ls -hl target/debug/librust_module.so 
-rwxr-xr-x 2 mathetake mathetake 23M Apr 16 01:58 target/debug/librust_module.so

[Xunzhuo]

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

Why not just directly add this to DM fields EEP API? Like what we did in WASM

[arkodg]

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

Why not just directly add this to DM fields EEP API? Like what we did in WASM

That will allow a route owner to access another route or even the listener violating its scope of extension

[mathetake]

So currently a dynamic module's capability is as same as Lua and Wasm so I don't have that concern right now. However yes indeed it can expand to have more broader capability plus a module won't be limited to HTTP filter (might end up having Load balancer extension etc). So loading/distribution of modules should be a separate API vs EEP which purely should focus on the configuration of HTTP filters. That's my 2c

[Xunzhuo]

That will allow a route owner to access another route or even the listener violating its scope of extension

Get it, thanks for clarifying @arkodg

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. Please feel free to give a status update now, ping for review, when it's ready. Thank you for your contributions!

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. Please feel free to give a status update now, ping for review, when it's ready. Thank you for your contributions!

[jukie]

@Xunzhuo any plans on picking this back up? Would love to see dynamic modules be supported

[arkodg]

@jukie the TODO with this one Is to figure out the delivery mechanism for the modules