disableWAL option

[tufitko]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Motivation

We are using kvrocks with client sharding, also we can lose data. So, WAL is useless for us and it just makes additional write pressure.

Solution

I made some changes to ensure about this: https://github.com/apache/incubator-kvrocks/commit/bb1ba2eb0e2897b17e73ec3af8320d82d4872a07

I can replace rocksdb::WriteOptions() to global WriteOptions with optional disabledWAL, what do you think about it?

Are you willing to submit a PR?

• [X] I'm willing to submit a PR!

[git-hulk]

Thanks for @tufitko feedback. So you don't need the replication, right?

I thinks we can add a new config option to let users to disable the WAL, and also need to forbid the slaveof command when the WAL is disabled since the replication can't work without it. How do you think?

[tufitko]

@git-hulk yeah, we don't need replicas. About slaveof sounds good, so I'm gonna make PR at the weekends

[git-hulk]

@git-hulk yeah, we don't need replicas. About slaveof sounds good, so I'm gonna make PR at the weekends

Cool, thanks. Can just reply error in the slaveof command if the WAL is disabled.

[aleksraiden]

What is the benefits from disabling WAL? This mechanism, as I remember, used for parallels writers, thats can work without blocking too. And, of course, data safety are poor. This only reduce a disk I/O, but in sequential mode (like WAL) I think, its no a lot of problem. maybe you want to reduce storage - you can disable wal history (in archive directory inside). Maybe this can help you quickly?

[tufitko]

@aleksraiden It's exactly for reducing a disk I/O. We have a lot of data to write, so any reduction of IO helps us.

[aleksraiden]

@tufitko, ok, interested to me too. I have a medium-load system (streams, ~20K rps, 100M+ msg in history, but 7.2.2 rocksdb) based on kvrocks, I will interest to test this patch too.