Global downstream connection limits.

[arkodg]

Add a overload manager configuration to limit the total number of downstream connections https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/overload_manager/overload_manager#limiting-active-connections

[tmsnan]

It appears that the functionality is incomplete and unsuitable for production use. Should we enable it? https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/resource_monitors/downstream_connections/v3/downstream_connections.proto#extensions-resource-monitors-downstream-connections-v3-downstreamconnectionsconfig

[arkodg]

@tmsnan im fine waiting, but atm we dont have any way of enabling overload manager

[tmsnan]

@arkodg If possible, I could first add other overload manager features that are already fully supported.

[arkodg]

@tmsnan sure, imo the others like heap size based will require an API addition, putting the burden of getting it right on the user . Was trying to use this GH issue to enable sensible defaults

[tmsnan]

@arkodg Maybe we can refer to Google VRP edge server configuration.

overload_manager:
  refresh_interval: 0.25s
  resource_monitors:
  - name: "envoy.resource_monitors.fixed_heap"
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.resource_monitors.fixed_heap.v3.FixedHeapConfig
      # TODO: Tune for your system.
      max_heap_size_bytes: 2147483648  # 2 GiB
  actions:
  - name: "envoy.overload_actions.shrink_heap"
    triggers:
    - name: "envoy.resource_monitors.fixed_heap"
      threshold:
        value: 0.95
  - name: "envoy.overload_actions.stop_accepting_requests"
    triggers:
    - name: "envoy.resource_monitors.fixed_heap"
      threshold:
        value: 0.98

https://www.envoyproxy.io/docs/envoy/latest/configuration/best_practices/edge#best-practices-edge

type overloadManager struct {	
	MaxHeapSizeBytes int64
}

Gateway will use overloadmanage (passing envoy configuration similar to Google VRP edge server configuration) when configuring overloadManager's MaxHeapSizeBytes, otherwise it will not be enabled. Is this method OK?

[arkodg]

yes this is the one I was referring to, which will require an API change, but we need to have a simple recommendation for the user for a value to set in this API. e.g. Use this value to set the approximate fixed heap value of your system, you can find that out by running this command "TODO"

[arkodg]

suggest using the value from best practices doc https://www.envoyproxy.io/docs/envoy/latest/configuration/best_practices/edge#configuring-envoy-as-an-edge-proxy

      overload:
        global_downstream_max_connections: 50000

[shahar-h]

Please assign me