raft-engine
raft-engine copied to clipboard
tikv
A persistent storage engine for Multi-Raft log
The idea is borrowed from RocksDB `bytes_per_sync`. It uses `sync_file_range`[^1] that allows asynchronously incrementally sync files to disk while they are being written. It smoothes out write I/Os over time...
Simple [benchmark](https://gist.github.com/tabokie/a290ec746a2fb887f55e0894362dcc84) shows more than 20% throughput improvement from direct I/O. This task is preferred to be stacked over #165.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start)...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Updates the requirements on [nix](https://github.com/nix-rust/nix) to permit the latest version. Changelog Sourced from nix's changelog. [0.28.0] - 2024-02-24 Added Added mkdtemp wrapper (#1297) Add associated constants UTIME_OMIT UTIME_NOW for TimeSpec...
Updates the requirements on [clap](https://github.com/clap-rs/clap) to permit the latest version. Changelog Sourced from clap's changelog. [4.5.0] - 2024-02-08 Compatibility Update MSRV to 1.74 [4.4.18] - 2024-01-16 Fixes (error) When lacking...
Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. Changelog Sourced from env_logger's changelog. [0.11.0] - 2024-01-19 Breaking Change Removed bespoke styling API env_logger::fmt::Formatter::style env_logger::fmt::Formatter::default_styled_level env_logger::fmt::Style env_logger::fmt::Color env_logger::fmt::StyledValue Removed...
Accidentally I found that raft-engine's data directory contains discontinuous file sequences in append queue: ``` 0000000000000003.rewrite 0000000000000846.raftlog 0000000000000847.raftlog 0000000000000848.raftlog 0000000000000849.raftlog 0000000000000850.raftlog 0000000000000851.raftlog 0000000000000852.raftlog 0000000000000853.raftlog 0000000000000854.raftlog 0000000000000855.raftlog 0000000000000856.raftlog 0000000000000857.raftlog 0000000000000858.raftlog 0000000000000859.raftlog...
If the header of the last log file is broken, and the there is no disk space, raft engine will not be able to start, because it requires rewriting the...
This PR introduces a hedged file system to double-write every io operation. The HedgedFileSystem manages two directories on different cloud disks. All operations of the interface are serialized by one...
In cloud environment, cloud disk IO may get stuck for a while due to cloud vendor infrastructure issues. This may affect the foreground latency dramatically. Raft log apply doesn't sync...