discord.js-modules
discord.js-modules copied to clipboard
feat: added `@discordjs/sharder` module
Please describe the changes this PR makes and why it should be merged:
I heard you wanted a nice and complete sharder. :eyes:
Sadly, this system is not compatible with [email protected], but it will be for a future version! This sharder draws inspiration from the strategy and plug-in system Sapphire has largely adopted over the months, the way Kurasuta handles messages and shards separately, and the structure of Discord.js's built-in sharding manager to make the update easy, alongside a few sprinkles of my own based on my experience when building a cross-process intercommunication system.
:warning: This PR is under heavy WIP: docs and tests are missing, they'll be done as I progress. The API may also change as new needs arise when building more code.
Let's talk about this sharder's structure! It is divided in three components:
- The Sharding Manager: it's the central class that handles the shards, from starting them up, to closing.
-
The Shards: they're strategies that define how the "channel" should be handled from the sharding manager, they can be of any kind, from workers (
WorkerShardHandler
) and forks (ForkProcessShardHandler
) to cross-server clusters going thru local clusters (ClusterProcessShardHandler
). Yes, you heard that. Although we'll most likely ship only the 3 local ones. You can make your own custom shard strategies. -
The Messengers: they're also strategies, but they define how data must be serialized and/or deserialized as well as handling message queues, we'll provide
RawMessageHandler
(data as-is, without converting to a Buffer or a string),JsonMessageHandler
(JSON), andBinaryMessageHandler
(V8 serder)... You can plug in your own ones, such as the Erlang External Term Format (ETF), YAML, TOML, etc.
This clear separation between responsibilities allow for greater customization of the core functionality of the library, reducing the need to use external libraries.
And coming in the future, we'll probably release The Message Mappers, which give a final touch-up to a message, such as appending a header/footer or even, using OpenSSL to encrypt messages over the network.
Issues to handle:
- ✔ DONE https://github.com/discordjs/discord.js/issues/2882: I'll change
respawn: boolean
torespawn: number
, accepting-1
andInfinity
for unlimited respawns, but a number that is decreased by one on each shard until it's ready, in which case it'll reset the counter. - https://github.com/discordjs/discord.js/issues/3560: I had an eureka moment here, it'll require a small refactor in Discord.js's WebSocket system but... turns out we can query
/gateway/bot
in the sharder, only once, and pass the data to the shards during spawn. We could also implement a query system so we can get that data from the ShardingManager without querying Discord more than once a day by caching and updating the result accordingly. - https://github.com/discordjs/discord.js/issues/3890: See above.
- https://github.com/discordjs/discord.js/issues/4095: Status unknown.
- https://github.com/discordjs/discord.js/issues/5364: The new ready handler should be a lot cleaner and far more efficient, but we'll see.
- https://github.com/discordjs/discord.js/issues/5581: Can it happen with this sharder? Let's find out.
Todo:
- [x] Consolidate the sharder's API, make everything more consistent.
- [x] Make
ProcessShardHandler
abstract, for the two classes below. - [x] Implement
ForkProcessShardHandler
. - [x] Implement
CusterProcessShardHandler
. - [ ] Implement
WorkerShardHandler
. - [x] Implement
RawMessageHandler
. - [x] Implement
BinaryMessageHandler
. - [x] Implement built-in cluster sharding (e.g. 2 processes with 5 shards each to have 10 shards) support.
- [x] Implement
fetchRecommendedShards
. - [ ] Implement shard client to interact with the shard manager,
ShardTunnel
?ShardClient
? - [ ] Type the events.
- [ ] Add documentation.
- [ ] Add tests.
Status and versioning classification:
- I know how to update typings and have done so, or typings don't need updating
- This PR changes the library's interface (methods or parameters added)
Codecov Report
Merging #87 (daf9d67) into main (c22081d) will decrease coverage by
0.08%
. The diff coverage isn/a
.
@@ Coverage Diff @@
## main #87 +/- ##
==========================================
- Coverage 89.03% 88.95% -0.09%
==========================================
Files 9 9
Lines 1487 1566 +79
Branches 153 162 +9
==========================================
+ Hits 1324 1393 +69
- Misses 12 13 +1
- Partials 151 160 +9
Impacted Files | Coverage Δ | |
---|---|---|
packages/rest/src/lib/REST.ts | 95.57% <0.00%> (-0.98%) |
:arrow_down: |
packages/rest/src/lib/errors/DiscordAPIError.ts | 81.81% <0.00%> (-0.66%) |
:arrow_down: |
packages/rest/src/lib/RequestManager.ts | 93.46% <0.00%> (-0.50%) |
:arrow_down: |
packages/rest/src/lib/utils/constants.ts | 100.00% <0.00%> (ø) |
|
...ackages/rest/src/lib/handlers/SequentialHandler.ts | 82.98% <0.00%> (+0.54%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update c22081d...daf9d67. Read the comment docs.