etcd-io
Interest for allocation-free reads
Is there community interest for allocation-free reads? This was a suggestion for future work from @benbjohnson.
After zooming into the existing code, it may be feasible.
It could be reached in 2 steps:
1 - Provide an allocation-free Get within an already allocated Tx transaction - i.e., a Tx.Get(keys ...[]byte) with variadic keys argument - note it doesn't rely on any Bucket allocation That first step gives us allocation-free batched reads
2 - Wrap it further into an allocation-free read outside of any Tx transaction (but still properly read-locking the db, etc...) through a DB.Get(keys ...[)byte) That second step offers us allocation-free atomic reads
If interest is sufficient, I may contribute some pieces
Judging by the proposed interfaces, is this supposed to ignore buckets entirely? The data model expects a bucket at this time-- would there be a "default" bucket? Is there any software right now that's running into performance problems from the allocations?
The place to start would be measuring the current Get cost with a golang benchmark using ReportAllocs.
This would entirely rely on the bucket structure - no default bucket. For example to retrieve the key "ccc" from bucket "bbb" inside bucket "aaa", you would perform a Get("aaa", "bbb", "ccc") I used strings to make the example simple but keys are in fact []byte of course. I agree this endeavor must be driven by a benchmarking approach.
I would be also interested in this. My current project uses multiple queries to display a page consisting of 20 objects (each has to be retrieved via a separate Get) and each of the objects requires some additional queries to join normalized data.
According to benchmarking, 33.05% of the time (cumulative) is spent in runtime.growslice() and the primary reason seems to be c.stack = append(c.stack, e) in Cursor.search(). Modifying bucket.Cursor() to use a sync.Pool and returning the stack back to the Pool at the end of bucket.Get() resulted in an immediate performance gain - 10840 ns/op vs. 13843 ns/op in my benchmark (48 allocs / op vs. 67 allocs/op). But there remains more work to be done, and the changes will probably introduce API changes like adding a Close() method to the cursor. My quick experiment just changed bucket.Get() and ignored all other places were cursors were used, including my project itself.
Alternatively, i wouldn't mind reusing cursor objects at the client side for multiple queries in different buckets (skipping the sync.Pool overhead).
