horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

feat: `WalReplicator` skeleton implementation

Open waynexia opened this issue 3 years ago • 0 comments

Signed-off-by: Ruihang Xia [email protected]

Which issue does this PR close?

ref #175

Rationale for this change

Follow the context from #175. ReaderTable needs to get the newest data from the corresponding WriterTable via WAL. This implements a replicator that keeps polling data from WAL.

This replicator works as following:

          register IDs
         need replicate
     ┌─────────────────────┐
     │                     │
     │              ┌──────▼───────┐
┌────┴─────┐        │  background  │
│Role Table│        │WAL Replicator│
└────▲─────┘        └──────┬───────┘
     │                     │
     └─────────────────────┘
          replicate log
            to table

A background replicate task is starting with Instance. Instance can register table to it with register_table(). The replicator will loop on the registered table to poll logs from WAL. Invalid table (those tables with incorrect states) will be removed from the queue.

There are two problems I can think of.

  • A table may be kicked out in accident without notification.
  • The entire replicate procedure inside a replicator is serialized.

What changes are included in this PR?

Implement the replicator described above.

Are there any user-facing changes?

No

How does this change test

New added unit test. The add/evict table logics need a RoleTable trait definition, I'm going to cover this part after we got one.

waynexia avatar Aug 08 '22 12:08 waynexia