hermes
hermes copied to clipboard
Consider changing current approach to rate limiting for filters
Currently, there are two separate rate limiting mechanisms:
- Rate limiting for messages filtering
- Rate limiting for messages delivery
Both share the same configuration (property Rate limit from the configuration of subscription).
With the current approach when we have a subscription where most of the messages are filtered out we may easily reach a filtering rate-limiter threshold. As a consequence, Hermes will slow down sending messages to the subscriber. This in turn may lead to a high lag even though the subscriber might handle more traffic.
Perhaps splitting configuration should help, but we have to be careful about processing too many messages for one subscription as it may cause the starvation of other subscriptions.
Related issue #656