hermes icon indicating copy to clipboard operation
hermes copied to clipboard

Consider changing current approach to rate limiting for filters

Open piotrrzysko opened this issue 4 years ago • 1 comments

Currently, there are two separate rate limiting mechanisms:

Both share the same configuration (property Rate limit from the configuration of subscription).

With the current approach when we have a subscription where most of the messages are filtered out we may easily reach a filtering rate-limiter threshold. As a consequence, Hermes will slow down sending messages to the subscriber. This in turn may lead to a high lag even though the subscriber might handle more traffic.

Perhaps splitting configuration should help, but we have to be careful about processing too many messages for one subscription as it may cause the starvation of other subscriptions.

piotrrzysko avatar Mar 23 '21 19:03 piotrrzysko

Related issue #656

druminski avatar May 20 '21 09:05 druminski