seafowl icon indicating copy to clipboard operation
seafowl copied to clipboard

Best effort chunk coalescing implementation

Open gruuya opened this issue 2 months ago • 0 comments

This is a naive implementation of https://github.com/splitgraph/seafowl/pull/514 (and therefore less complex), where we don't use synchronization to avoid request de-duplucation, but instead try do a best-effort chunk coalescing, and batch them into a single call.

Compared to current main, this implementation seems to be more stable, and doesn't experience the hanging in TPC-H q18 and q21, since it does not use any synchronization primitives.

As for perf, in the (idealized) no latency case it is on par with the current caching object store. However in more realistic scenarios of modest delays (15ms) it consistently outperforms it: image

gruuya avatar Apr 16 '24 07:04 gruuya