electric icon indicating copy to clipboard operation
electric copied to clipboard

Consumer.Monitor.RefCounter is slowing down shape deletion

Open magnetised opened this issue 2 months ago • 0 comments

In order to prevent errors from reads to a shapes files we have a process that waits until all api read requests for a shape have terminated before terminating the shape consumer and deleting its data.

This requires serialising all deletions through a single genserver that tracks termination requests and api requests which is a major bottleneck when deleting lots of shapes.

Instead of this I propose simply delaying the deletion of the shape's data by some number of minutes (say 5). This will ensure that all active reads will have terminated (with nearly 100% probability) before deleting the data without the need to track state in a genserver.

magnetised avatar Nov 05 '25 15:11 magnetised