redb icon indicating copy to clipboard operation
redb copied to clipboard

Support concurrent writes to disjoint tables in the same transaction

Open cberner opened this issue 3 years ago • 2 comments

cberner avatar Dec 30 '21 23:12 cberner

This does not look promising from benchmarking two concurrent database writes. Running the lmdb_benchmark script in two threads leads to:

redb: Bulk loaded 100000 items in 964ms
redb: Bulk loaded 100000 items in 970ms
redb: Wrote 100 individual items in 1235ms
redb: Wrote 100 individual items in 1236ms
redb: Wrote 100 x 1000 items in 6547ms
redb: Wrote 100 x 1000 items in 6547ms
redb: Random read 100000 items in 60ms
redb: Random read 100000 items in 61ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 65ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Removed 50000 items in 415ms
redb: Removed 50000 items in 418ms

vs a single thread:

redb: Bulk loaded 100000 items in 457ms
redb: Wrote 100 individual items in 776ms
redb: Wrote 100 x 1000 items in 4139ms
redb: Random read 100000 items in 60ms
redb: Random read 100000 items in 54ms
redb: Random read 100000 items in 54ms
redb: Random range read 100000 starts in 63ms
redb: Random range read 100000 starts in 63ms
redb: Random range read 100000 starts in 63ms
redb: Removed 50000 items in 257ms

showing no gain for bulk inserts, and only a ~1.5x throughput improvement for fsync bound workloads

cberner avatar Jan 15 '22 22:01 cberner

Seems worth taking another look at this given the low throughput findings for large databases in #344

cberner avatar Sep 11 '22 05:09 cberner