JustAnotherArchivist
JustAnotherArchivist
There isn't any benefit to that over just disabling flushing entirely though. The Raft spec requires that nodes do not respond to RPCs before persisting those values to disk. So...
Ah, so the `append_entries` messages typically already have multiple entries? In that case, it should be possible to make `SyncObj` append multiple entries at once to the journal and only...
Ok, that helps, but unfortunately the batches seem to be rather small (less than 10 entries in my tests), so it only provides a fairly small improvement. My test machine...
Yeah. It looks like the `Journal.add` call in `SyncObj._checkCommandsToApply` is the bottleneck. After disabling flushing there, I manage 12k RPS. (That's with the batch-addition to the journal in the `append_entries`...
I'm not sure what the best solution for the `votedFor` issue is. This would also be affected by the network rewrite I'm currently working on (#4), so it might make...
Thanks. 1. That might work, though I'm not entirely sure if there are any corner cases to it. I think it's better to follow the Raft spec and persist the...
1. Yes, that sounds good. I looked a bit at how the journal works and have an idea on how to implement this. The `votedFor` variable would move to `Journal`...
Since #92 is merged, I've been looking into this, and I realised that I missed something above: the current term also needs to be persisted, not only the voted-for node...
By the way, I found [a discussion](https://groups.google.com/d/msg/raft-dev/yzT3JTZweH0/xfPbCSwsAwAJ) on this topic on the raft-dev mailing list, which also mentions the idea of waiting for the maximum election timeout on boot to...
I just noticed that some files `dump1.bin.1.tmp` etc. are left over in the PySyncObj directory after these test failures. The `tmp` extension again suggests that the issue is related to...