apache
feat: support maxmemory-clients config
relate to https://github.com/apache/kvrocks/issues/2284 Implemented a new maxmemory-clients configuration to enable client eviction based on overall memory usage, preventing potential OOM issues due to excessive client connections.
I want to discuss how we can implement this feature. My idea is to count client memory within each worker (using libevent's callback to count evbuffer) to avoid locking across workers. @AntiTopQuark @PragmaTwice @git-hulk @mapleFU @torwig
I want to discuss how we can implement this feature. My idea is to count client memory within each worker (using
libevent's callback to countevbuffer) to avoid locking across workers. @AntiTopQuark @PragmaTwice @git-hulk @mapleFU @torwig
Yes, as mentioned in previous comment. It would be better to limit the connection output first by checking the libevent buffer. And for the maxmemory-clients, I think we don't need to count every byte, just checking the libevent input and output buffer is good to me as well.
I want to discuss how we can implement this feature. My idea is to count client memory within each worker (using
libevent's callback to countevbuffer) to avoid locking across workers. @AntiTopQuark @PragmaTwice @git-hulk @mapleFU @torwigYes, as mentioned in previous comment. It would be better to limit the connection output first by checking the libevent buffer. And for the
maxmemory-clients, I think we don't need to count every byte, just checking the libevent input and output buffer is good to me as well.
I have modified the code and moved the evictionClients function into Worker.
On another note, I believe that maxmemory-clients is used to limit the total size of connections. If you want to limit the size of the output buffer, you can use the client-output-buffer-limit configuration option. I will submit a PR for this in the future.
want to discuss how we can implement this feature. My idea is to count client memory within each worker (using libevent's callback to count evbuffer) to avoid locking across workers.
Yeah I think holding a lock is harmful here, I can accept some untimely memory change, but global checking is harmful...
I believe implementing an elegant solution to track and restrict client connection memory usage is challenging and could reduce code maintainability.
Instead, I suggest exploring alternative approaches, like limiting the number of clients.
I believe implementing an elegant solution to track and restrict client connection memory usage is challenging and could reduce code maintainability.
Instead, I suggest exploring alternative approaches, like limiting the number of clients.
I believe implementing an elegant solution to track and restrict client connection memory usage is challenging and could reduce code maintainability.
Instead, I suggest exploring alternative approaches, like limiting the number of clients.
When I looked at Redis previously, it uses the maxclients configuration option to limit the number of clients, and maxmemory-clients to limit the total memory used by connections. If you only want to limit the output buffer, you can use client-output-buffer-limit to restrict the usage per connection.
@PragmaTwice @git-hulk Hi, should we continue pushing this PR forward, or can it be closed?