forest icon indicating copy to clipboard operation
forest copied to clipboard

Rate-limiting for the RPC API

Open lemmih opened this issue 2 years ago • 1 comments

Issue summary

Lotus has a separate executable for providing safe access to their API. We could re-use their gateway executable to provide safe access to our API, but it would give a smoother experience if we directly supported rate-limiting in Forest. With built-in rate-limiting, no gateway would be needed.

Our flags should be similar to those in the Lotus gateway but may deviate for ergonomics.

Limits:

  • [ ] Maximum lookback as a duration. Some calls (like MpoolPending) take a Tipset as an argument. Access to tipsets older than the maximum lookback should be rejected. Default value: 24 hours. Lotus flag --api-max-lookback.
  • [ ] Maximum request size. JSON requests larger than the maximum request size should be rejected. Default value: disabled. Lotus flag --api-max-req-size.
  • [ ] Message lookback limit in epochs. Some calls look for messages in past tipsets. Tipsets further away from the HEAD than this lookback limit should not be searched. Default: 20 epochs. Lotus flag --api-wait-lookback-limit.
  • [ ] Maximum number of connections per IP per minute. Default: disabled. Lotus flag --conn-per-minute
  • [ ] Maximum number of RPC calls per second per connection. Default: disabled. Lotus flag --per-conn-rate-limit.
  • [ ] Maximum number of RPC calls per second (global). Default: disabled. Lotus flag --rate-limit.
  • [ ] Rate limit timeout. Number of seconds to wait for available capacity before returning an error. Default: 5s. Lotus flag --rate-limit-timeout.

Figuring out how to test these limits is a big part of this task.

Other information and links

Lotus Gateway: https://lotus.filecoin.io/lotus/configure/gateway/

Go rate: https://pkg.go.dev/golang.org/x/time/rate

lemmih avatar Oct 09 '23 12:10 lemmih

  • See also https://github.com/ChainSafe/forest/issues/3029

aatifsyed avatar Oct 10 '23 14:10 aatifsyed