Initial Config Manager Design and Implementation

[arkodg]

The Config Manager is responsible for instantiating and managing the lifecycle of the various system components in Envoy Gateway based on the Envoy Gateway Bootstrap spec

[arkodg]

Here are some options for lifecycle management of system components

• https://github.com/tetratelabs/run based off https://github.com/oklog/run/ . Here's an example
• https://github.com/datawire/dlib/tree/master/dgroup . Here's an example

[danehans]

@youngnick @skriss how does Contour handle config management?

[youngnick]

Our Gateway API support works like this:

• Contour has a CRD (ContourConfig) that configures an individual xds server and associated Envoy fleet.
• The Gateway provisioner has a CRD (ContourDeployment) that configures how to deploy an xds server and associated Envoy fleet. This includes a ContourConfig inline in the ContourDeployment spec, so that when we add to ContourConfig, the ContourDeployment gets the changes automatically.
• The ContourDeployment is referenced in the GatewayClass paramsRef section, and the new xds servers are stamped out as Gateways are created. Note that changes in the ContourDeployment are not reconciled to already-existing xds server installs.

Both the xds server and the provisioner are separate subcommands of the main contour binary; they are run in separate containers (so that the provisioner works by creating a new namespace, xds server deployment, and Envoy proxy daemonset or deployment).

I like this method because it means that the provisioner just needs to be a Kubernetes controller whose job it is to make sure that xds servers and Envoys are provisioned as directed by Gateways, rather than needing to reinvent the job-control wheel by managing multiple xds servers in the same container. Kubernetes has already solved the job-control problem, why not use it as much as possible?

[arkodg]

great thanks for sharing that ! I completely agree, we should allow the user to run xdS Translator + Server separately from the provisioner if they wanted to, mainly to allow fine grained auto scaling of different components. In the runGroup case, the Config Manager can decide which services (provisioner and/or xds Server) to instantiate based on Gateway Bootstrap Config ( If the provisioner spec/fields are missing/disabled, the component will not be instantiated). This approach adds optionality of running all components as one process if needed (mainly useful for standalone VM controller cases or app dev local dev workflow testing) or as separate processes/containers in production.

[arkodg]

depends on https://github.com/envoyproxy/gateway/issues/51

[danehans]

@arkodg I assigned this issue to you based on the 5/24/22 community meeting notes. Please unassign yourself if that's not the case.

[danehans]

Although the initial Config Manager will focus on k8s, it should be designed to support add'l config sources, xref: https://github.com/envoyproxy/gateway/issues/30

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[danehans]

Fixed by https://github.com/envoyproxy/gateway/pull/146

[danehans]

Reopened by https://github.com/envoyproxy/gateway/pull/197

[danehans]

@arkodg can this issue be closed?

[arkodg]

sure lets close, we seem to be doing fine w/o any sophisticated config manager atm, lets reopen if the need arises.