[Feature] RPC rate limiter

[howardwu]

🚀 Feature

1. Add controls that allow snarkOS to adjust 2 rates: a. The rate that RPC requests from all IP addresses are processed b. The rate that RPC requests from a single IP address is processed

2. Add a "restricted_peers" list for RPC IP addresses that exceed the rate.

Motivation

Some IP addresses are spamming the sync node RPC endpoints.

[ljedrz]

Note: jsonrpsee just declared that it's more or less stable a few days ago; I'll check it out, it might be the time to close https://github.com/AleoHQ/snarkOS/issues/754 as well.

[ljedrz]

Oh, and just so that no one else tackles it in parallel, I'm already working on this.

[howardwu]

What's the status on this?

[ljedrz]

Let's consider the the things we've used the RPC server for so far:

• obtaining various information on our own node
• actively interacting with our own node
• providing data for the Aleo explorer from the a dedicated Aleo node
• taking some measurements of the entire network

And some of the related issues we've encountered:

• sync nodes (and possibly any other node with their RPC server open to the network) being overwhelmed with the volume of RPC requests, slowing them down
  • [x] resolved by setting a global jsonrpsee resource limit
• sync nodes were sometimes getting a lot of requests from specific hosts (indicating malicious intent), slowing down the rate at which legitimate requests could be processed
  • [ ] open issue (the remaining part of this very issue)
• many nodes have had their RPC server closed/unreachable, causing network-wide measurements to be only partial
  • [x] mitigated by the freshly merged network-based crawler, which will be extended with further features
• some users didn't want to expose the RPC server at all (e.g. #1355) due to security concerns
  • [ ] partially mitigated by using a dedicated RPC server implementation (as opposed to a "hand-written" one), but still open

Having all the above in mind, I'd like to propose to start shifting towards dedicating the RPC server to local/restricted use; this means:

• setting the default RPC server IP address to 127.0.0.1
• using jsonrpsee's AccessControl to restrict the hosts that can send RPC requests to the sync nodes (this would be employed by the nodes we want to use for Aleo explorer's purposes)
• only using the network-based crawler to obtain much more complete information on the whole network (while also distributing lists of peers)

This proposal would completely resolve any concerns related to malicious RPC use; even if we used per-IP bans, a well-organized malicious actor could still set up a lot of RPC clients that would individually send requests at a rate right above the one we permit.

Cc https://github.com/AleoHQ/snarkOS/issues/1381

[ljedrz]

One extra point for limiting RPC access as a policy: if we add private endpoints again, we would need to have a way of securely inputting and transmitting passwords in the headers, otherwise they'd be visible in command line history and transmitted in plaintext, which is very insecure.

Cc https://github.com/AleoHQ/snarkOS/issues/1583