durable-queue icon indicating copy to clipboard operation
durable-queue copied to clipboard

Slabs can grow larger than max-queue-size

Open sundbry opened this issue 7 years ago • 0 comments

In the put! function, slab! is called before queue!, writing the data to the slab before offering to the queue.

               (if-let [val (locking q
                              (queue!
                                (vary-meta (slab!) assoc
                                  ::this this-ref
                                  ::queue-name q-name
                                  ::queue q
                                  ::fsync? fsync-take?)))]

The consequence is that the slab files can get larger than the max-queue-size, and upon application restart, exceptions get raised when the slabs are loaded and don't fit into the queue.

(IllegalArgumentException.
                             "'max-queue-size' insufficient to hold existing tasks."))

sundbry avatar Apr 25 '18 20:04 sundbry