bookkeeper icon indicating copy to clipboard operation
bookkeeper copied to clipboard

Issue #4033: Added a flush function to run flush without commit

Open YutSean opened this issue 2 years ago • 6 comments

Descriptions of the changes in this PR:

Added a sync method for AppendOnlyWriter

Motivation

We are testing using Distributedlog as HBase WAL storage. For HBase, there are three different durability levels, async wal, sync wal, fsync wal. The features of these three kinds of wal are:

async wal: the main thread only invokes append (write method of appendonlywriter) and returns.

sync wal: the main thread returns after confirming that the data is already arrived at the storage service through network.

fsync wal: the main thread returns after confirming that the data is already persistently stored.

But currently, the AppendOnlyWriter only provides a method force, which provides a stronger durability than fsync wal (the client will wait until the data is visible to other clients).

Stronger durability means poorer performance. The purpose to provide three different WALs is to make it possible for clients to make a trade off between durability and performance.

Changes

Added a new method flush of AppendOnlyStreamWriter, which just invoke the underlying flush#BKLogSegmentWriter and to wait the future completed or not by a boolean parameter.

Master Issue: #4033

YutSean avatar Jul 17 '23 09:07 YutSean