How can I use the ratelimit setting dynamically?

[jbj616]

I want to change config info by my server, Is there any other way than to modify the config.yaml file?

[dweitzman]

At the moment seems like you'd have to modify config.yaml. I'm also interested in the idea of a pluggable quota interface that could query an external database.

A yaml file works ok when you have a small number of quotas, but doesn't seem to scale as well when you have thousands or millions of clients. Seems like eventually a person would want policies for how quotas are automatically set or adjusted, and policies about who can request quota increases (or decreases) and how those requests get automatically or manually reviewed.

[mattklein123]

I think it makes sense to allow plugabiity of the configuration.

I think the correct thing to do here would be to actually have an xDS API that allows rate limit instances to talk to a management server. (Somewhat meta, but I think it's actually what we need). cc @envoyproxy/api-shepherds as this is an interesting case of a service as a client.

[htuch]

Does the extension config DS serve this purpose?

[mattklein123]

Does the extension config DS serve this purpose?

Probably yes, but would need some research. This would also be a good opportunity to have a factored out Go based xDS client (cc @jessicayuen @markdroth)

[ikingye]

Does the extension config DS serve this purpose?

https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/extension/v3/config_discovery.proto

[ankorGH]

hey @mattklein123 , what is the status of this issue. With some guidance, I will like to take this on.

[mattklein123]

hey @mattklein123 , what is the status of this issue. With some guidance, I will like to take this on.

I don't think anyone is working on this. Sorry but I don't have time to provide more guidance than what is already in the issue. We need to design and propose an API, probably based on xDS, that would allow the service to fetch new configurations.

[renuka-fernando]

hey @mattklein123, I would like to contribute to this. Is there any design for this feature?

As a starting point I have come up with some sample proto and configuration in settings.

1. Configuration

CONFIG_TYPE="GRPC_XDS_SOTW" # "FILE|GRPC_XDS_SOTW"
CONFIG_GRPC_XDS_SERVER_URL="localhost:19001"
CONFIG_GRPC_XDS_SERVER_USE_TLS="false"
CONFIG_GRPC_XDS_SERVER_TLS_CERT=""
CONFIG_GRPC_XDS_SERVER_TLS_KEY=""
CONFIG_GRPC_XDS_SERVER_TLS_CACERT=""
CONFIG_GRPC_XDS_SERVER_TLS_SAN="localhost"

# some other advanced configs
CONFIG_GRPC_XDS_SERVER_CONNECT_TIMEOUT="1s"
CONFIG_GRPC_XDS_SERVER_TCP_KEEP_ALIVE_PROBE="3"
CONFIG_GRPC_XDS_SERVER_TCP_KEEP_ALIVE_TIME="300"
CONFIG_GRPC_XDS_SERVER_TCP_KEEP_ALIVE_INTERVAL="30"

2. Message

syntax = "proto3";
 
package ratelimit.service.config.v1;
 
option go_package = "github.com/envoyproxy/go-control-plane/ratelimit/service/config/v1;configv1";
 
// [#protodoc-title: Rate Limit Config]
 
// Rate-limit config model
message RateLimitConfig {
    string domain = 1;
    repeated RateLimitDescriptor descriptors = 2;
}
 
// Rate-limit descriptor model
message RateLimitDescriptor {
    string key = 1;
    string value = 2;
    RateLimitPolicy rate_limit = 3;
    repeated RateLimitDescriptor descriptors = 4;
    bool shadow_mode = 5;
}
 
// Rate-limit policy model
message RateLimitPolicy {
    string unit = 1;
    uint32 requests_per_unit = 2;
    bool unlimited = 3;
    string name = 4;
    repeated RateLimitReplace replaces = 5;
}
 
// Rate-limit replace model
message RateLimitReplace {
    string name = 1;
}

3. Service

syntax = "proto3";
 
package ratelimit.discovery.service.config;
 
import "envoy/service/discovery/v3/discovery.proto";
 
option java_package = "io.envoyproxy.ratelimit.service.config.v1";
option java_outer_classname = "RlsConfigProto";
option java_multiple_files = true;
option java_generic_services = true;
option go_package = "github.com/envoyproxy/go-control-plane/ratelimit/service/config/v1;configv1";
 
// [#protodoc-title: RateLimitConfigDiscoveryService]
service RateLimitConfigDiscoveryService {
    rpc StreamDescriptors(stream envoy.service.discovery.v3.DiscoveryRequest)
        returns (stream envoy.service.discovery.v3.DiscoveryResponse) {
    }
    
    rpc FetchConfigs(envoy.service.discovery.v3.DiscoveryRequest) returns (envoy.service.discovery.v3.DiscoveryResponse) {
    }
}

I guess we can include the server implementation within go-control-plane as we could reuse the most of the existing implentation there.

I would like to know your thoughts on this.

[mattklein123]

Sure ^ sounds like a good approach to me!

[renuka-fernando]

Hey @mattklein123, I am proposing a new interface RateLimitConfigProvider and hope to move file config watcher (runtime) from service to ConfigProvider.

In current implementation Service holds the file watcher (runtime), when a config is changed it calls the ConfigLoader.

The new interface RateLimitConfigProvider which is responsible for watching for config changes (from file, xDS gRPC server or Rest Server) and send new config back to the Service.

package config

type RateLimitConfigProvider interface {
	ConfigUpdateEvent() <-chan *ConfigUpdateEvent
}

type ConfigUpdateEvent struct {
	Config RateLimitConfig
	Err    any
}

File Config Provider

xDS Config Provider

[renuka-fernando]

To implement the xDS client for rate limit service we need a sample xDS server to test the functionality. The xDS server implementation would be in go-control-plane repository https://github.com/envoyproxy/go-control-plane/issues/595. Proto files should be added to this repository (ratelimit repository).

Hence I thought of doing the implementation in the following order.

1. Adding proto files to the rate limit repository (PR: https://github.com/envoyproxy/ratelimit/pull/368)
2. Server-side implementation within go-control-plane repository (PR: https://github.com/envoyproxy/go-control-plane/pull/598).
3. Complete configuration processing in rate limit service via xDS (TODO)

[markdroth]

I agree that it would be nice to have a generally useful xDS client API for Go. (Actually, it would be good to have one for all languages, but I realize that we're talking specifically about Go here.)

@dfawley @easwars Once we're done with the resource-type-agnostic XdsClient API changes in grpc-go, how hard would it be to pull the XdsClient out of the gRPC code so that it could be used by other clients?

[dfawley]

This topic has come up before and IIRC we decided to punt on it.

Technically, it's not too hard to split it out, since it's in a separate package. However, it's a global singleton which might not be ideal as a separable package, and it would need a stable API, which would require additional consideration and could make future development more difficult. Our code is open source; if a user needs this, they can copy it to their own repo and use or modify it as needed as long as they follow the license terms.

[easwars]

I agree with @dfawley that moving this out would mean that we would have to make the API stable, and would make future development difficult. But after the generic API changes are done, if we don't expect large scale changes to this API, we can consider doing this.

On the other aspect of it being a singleton, we can continue to retain the singleton code in its current place, and consider moving out only the actual xDS client implementation.

[carnei-ro]

I like the solution proposed by @renuka-fernando; until it is not merged/released, we are using consul-template to edit the configuration file dynamically. (Using the env var RUNTIME_WATCH_ROOT set to false)

[kkcmadhu]

I simply took the docker-compose.yml shipped with this projcted , added env variable RUNTIME_WATCH_ROOT = false and changing the config file manually, is not working for me :( refer https://github.com/envoyproxy/ratelimit/issues/379

for any xds,ads solution, it will be super cool if this can be added to both go and java control planes.

[renuka-fernando]

Thank you all for your reviews, and suggestions. @alecholmez, thank you for your support on the go-control-plane side to get this feature done.

[kkcmadhu]

thanks much @renuka-fernando for this implementation, can you kindly include the same in java control plane too, that will be really useful.