gloo icon indicating copy to clipboard operation
gloo copied to clipboard

[1.12] gRPC plugin not being reset correctly - causing gloo performance and possibly envoy memory spike

Open saiskee opened this issue 2 years ago • 0 comments

Gloo Edge Version

1.12.x (latest stable)

Kubernetes Version

1.24.x

Describe the bug

When backporting the change to instantiate plugins only once (https://github.com/solo-io/gloo/pull/7104), I needed to backport the fix in which we make sure that all plugins are using Init() correctly, as this resets the state of the plugins. However, since the latter fix was never backported, the gRPC plugin's state is never reset correctly, and as routes update, config builds up in size. Because this config includes large gRPC descriptor protos, this can slow down envoy's config-time loading and possibly cause a memory spike in envoy.

Steps to reproduce the bug

On gloo oss/enterprise version v1.12.15, create a route that points to some gRPC upstreams that we have done gRPC function discovery on. Then kick a pod a few times so our entire translation loop runs, and you'll notice that every time the pod is kicked, envoy also recieves new listener configuration.

Expected Behavior

na

Additional Context

No response

saiskee avatar Sep 16 '22 15:09 saiskee