kord
kord copied to clipboard
Multiprocess support
Summary
Create Remote
implementations of Gateway
, RequestHandler
and DataCache
that can comminucate rate limits between multiple Kord processes.
Goals
Make it viable for one bot token to be used over multiple Kord processes by:
- syncing gateway send ratelimits
- syncing gateway indentify ratelimits
- syncing rest ratelimits
- syncing cache data
Non-Goals
This issue does not intend to alter existing code (unless necessary), the implementation of this feature should be its own module.
Motivation
Kord technically supports multiprocess applications, but there is no support for the above goals. While this is a setup that won't be applicable for most users, it is something we should support if we wish to call Kord feature complete.
Description
Kord allows a single process to only run a subset of the suggested/required shards, but there is no support for the syncing of resources between these separate processes. This feature request intends to implement the needed parts to make this scenario relatively hassle-free.
The approach suggested is to create a RemoteGatewayClient
, RemoteRequestHandlerClient
, RemoteDataCacheClient
and RemoteRateLimiterClient
and their respective RemoteXServer
variants, where clients will, based on a stream protocol, communicate the needed information with each other to remain in sync.
For everything but the DataCache, this would simply involve requesting and updating ratelimits with the server, requesting when the next action can be executed, notifying the consumption of tokens and information related like bucket rate limits. The DataCache
however would require the ability to store and fetch complex data structures.
Conceptually, we're looking for something like rsocket to implement the protocol layer, although the requirement on netty, rxjava and other jvm-only libraries would mean that we'd have to find another implementation for our eventual move to multiplatform. We could simple write our own protocols on top of ktor, but the work needed for this might be out of scope. This issue will be expanded upon once we have found a suitable protocol layer.
Issue originally made by @BartArys
I like this challenge, is there any plan over this?
I have a few questions related to this:
- Which communication protocols are you expecting to implement for this feature as mvp?
- Are you planning on using a custom protocol for this purpose? If so is it worth rather than implementing another protocol?
- Is there any discussion/plan on the required components to achieve this? that way maybe it can be split up into smaller pieces.