Module shuffling and organizing

[Hoverbear]

Reorganize some of the modules to have their own files and directories.

In general:

• Files should contain no more than one logical unit. (Eg. the RawNode struct and implementations.)
• Files should contain in order:
  • Module level docs (if existing)
  • use statements.
  • Any constants. (Eg INVALID_ID)
  • Any short utility functions that are not associated with the structure.
  • Any minor structures/enums that are trivial and don't warrant their own file. Eg enum Foo { Bar, Baz }. (If it has implementations beyond 1-2 functions split it off)
  • The structure itself.
  • Any implementations for the structure.
  • A test module if it's small (less than 5 tests, otherwise make it a $NAME/{mod, tests}.rs pair).

Changes for this PR should only include moving code and updating related use statements etc. No logic changes should happen here, they deserve their own PR/issue. This is to ease review.

Rationale

In TiKV we've been seeing a lot of long build times and one optimization to this is splitting up large files. Since raft is a considerably smaller crate it's easy to test our ideas here. At the same time, Raft has some very large files which may be both intimidating and confusing to navigate for potential contributors.

Benefits

• No run time performance changes are expected.
• Build time performance should strictly improve.
• Increased ease of navigation

Process

In some cases, we should to split files because they hold large structure definitions that can be split up. For example, src/progress.rs contains the structs Progress and ProgressSet , as well as Inflights.

Split modules with several non-trivial structures. In our example, src/progress.rs becomes src/progress/{mod, progress, progress_set, inflights}.rs

There are also modules that are very large due to large test suites etc. They can be split up. For example, src/raft.rs contains a large #[test] module.

Extract large test suites into their own files. In our example all of the #[test] code can go in a tests file. src/raft.rs becomes src/raft/{mod, tests}.rs.

[Hoverbear]

@breeswish I know you looked at build performance on TiKV, maybe you can have a look and give a LGTM if you think it's a good thing to do?

[breezewish]

I like this proposal! For TiKV, currently one of the slow cause might be rust-rocksdb, which requires a full fresh build of rocksdb and spend a lot of time.

[Hoverbear]

@breeswish Okay. :) I'll start to migrate Raft-rs in this direction over time. Other PRs are welcome to contribute to this!

[Fullstop000]

@Hoverbear I'll start to spilt src/progress.rs into mods after 0.5.0 is released. And this will also be a preparing for Follower Replication implementation.

[Hoverbear]

@Fullstop000 That's great! Thanks!

Do you think we can write up some technical details about Follower Replication? I think it's deserving of a full TiKV RFC (https://github.com/tikv/rfcs/). I'd be happy to help you with that. Maybe we can connect on wechat?

[Fullstop000]

Oh it's so cool having a chance to propose a RFC about it. But I'm still considering about details of the implementation, maybe we can discuss this later.

[Hoverbear]

@Fullstop000 It is very exciting! Why don't you let me know when you've considered and we can set up a call to discuss.