raft-engine icon indicating copy to clipboard operation
raft-engine copied to clipboard

Support asynchronously incrementally sync files to disk while they are being written

Open overvenus opened this issue 1 year ago • 4 comments

The idea is borrowed from RocksDB bytes_per_sync. It allows asynchronously incrementally sync files to disk while they are being written. It smoothes out write I/Os over time and reduce fdatasync latency spike as we saw in https://github.com/tikv/raft-engine/pull/346#issue-2059165690 .

Note: The feature proposed here is different from the deprecated bytes_per_sync feature which sync file synchronously.

overvenus avatar Jan 03 '24 10:01 overvenus

A similar issue: https://github.com/GreptimeTeam/greptimedb/issues/3085 Currently we use periodical fsync to alleviate the problem.

Before the periodical fsync the disk ioutil looks like: image

v0y4g3r avatar Jan 10 '24 11:01 v0y4g3r

THx for your advices. This problems might not be similar to yours. And we've met this problem on Cloud Env when rewriting and fixed it in #347, by introducing bytes_per_sync to limit the maximum flushing bytes. /cc @v0y4g3r

FYI, u can check whether there exists similar syncing problems when rewriting raft logs. And if does, u can check the latest version as your choice.

LykxSassinator avatar Mar 12 '24 08:03 LykxSassinator

THx for your advices. This problems might not be similar to yours. And we've met this problem on Cloud Env when rewriting and fixed it in #347, by introducing bytes_per_sync to limit the maximum flushing bytes. /cc @v0y4g3r

FYI, u can check whether there exists similar syncing problems when rewriting raft logs. And if does, u can check the latest version as your choice.

Do you have any plan to release a new version?

v0y4g3r avatar Apr 16 '24 08:04 v0y4g3r

/cc @overvenus @tabokie PTAL, thx.

We'll do it recently. Before that, u can update your dependency of raft-engine with the latest commit-id.

LykxSassinator avatar Apr 16 '24 12:04 LykxSassinator