orion-server icon indicating copy to clipboard operation
orion-server copied to clipboard

manage memory utilization of query executor

Open cendhu opened this issue 3 years ago • 0 comments

We need to control the total amount of memory used by the query executor to avoid OOM error. As each query collects keys and also values, the amount of memory used could go beyond the total RAM size and it could result in the panic of the bcdb node.

Hence, we need to introduce memory limit for the query executor and ensure that the in-memory data does not go beyond this limit. When the limit reaches, we can store the partial results to the disk and control the memory usage as done in PostgreSQL. When we address the issue, we can add more design details here.

cendhu avatar Sep 15 '21 08:09 cendhu