influxdb icon indicating copy to clipboard operation
influxdb copied to clipboard

Ensure buffer indicates backpressure to writing clients

Open pauldix opened this issue 2 years ago • 0 comments

Part of the configuration of the buffer will be to indicate how much memory it should use at most to ensure the server doesn't get OOM killed. If the buffer is filling up faster than it can compact data into Parquet and persist, then it must be able to indicate that it is overloaded.

There should be some number that is tracked that indicates how backed up the buffer is with compaction and persisting tables. This should be a metric exposed in /metrics. The server should have access to this number to determine if it should reject writes or hold them for some small period of time to slow down the writers.

pauldix avatar Jan 12 '24 20:01 pauldix