acid
acid copied to clipboard
Generic group commit implementation
LMDB (and similar) are single-writer engines, meaning multiple writers will cause contention and generally pointless wasted/blocked threads. Since LMDB supports sub-transactions, it's possible to do better.
Not sure what this should look like. Not sure if it should be integrated with the API somehow. For example, by changing Store.in_txn() decorator to Store.in_write_txn(), we could swap Store out for a subclass that schedules the function to execute on a writer thread, as part of a larger write transaction.
That may not be useful. The most interesting part of a generic implementation is not having to write an ad-hoc class when using e.g. gevent. Needs a lot more thought