middleware icon indicating copy to clipboard operation
middleware copied to clipboard

NAS-129692 / 24.10 / rate limit methods that have @no_auth_required

Open yocalebo opened this issue 1 year ago • 2 comments

For obvious reasons, we MUST protect public endpoints that are marked with @no_auth_required by rate limiting them. This implementation will rate limit all methods that are marked with @no_auth_required. These rate limits are implemented by hashing them by method name AND the origin IP address that is sending the request to us. This isn't a fool-proof way and won't cover every single scenario but it's significantly better than what we currently have.

Implementation details:

  1. ignore the rate limit if the method is being called across the unix socket (internal)
  2. ignore the rate limit if the method is being called via the heartbeat interface on HA systems
  3. store the method name and ip address of the remote client making the request
  4. if a method that doesn't require authentication has been called by the same origin IP address more than 20 times within a 1min period, the rate limit will kick in
  5. if the rate limits are being enforced, we will sleep a random delay to provide some form of backpressure to the offending client
  6. if the global cache limit has been reached (100 total entries), requests made by ANY remote caller to methods that don't require authentication will be blocked
  7. the global rate limit cache will be cleared every 10mins

yocalebo avatar Jun 21 '24 19:06 yocalebo

Jira URL: https://ixsystems.atlassian.net/browse/NAS-129692

bugclerk avatar Jun 21 '24 19:06 bugclerk

We used to have a @throttle decorator for somethings. Dunno if the intent was to handle that later or you could remove that as part of the PR, e.g. system.build_time

william-gr avatar Jun 21 '24 19:06 william-gr

This PR has been merged and conversations have been locked. If you would like to discuss more about this issue please use our forums or raise a Jira ticket.

bugclerk avatar Jul 05 '24 19:07 bugclerk