QinZuoyan

功能：对建立了replication关系的两个表进行数据验证，保证数据的一致性 使用场景：对于建立了replication关系的两个表，可以定期（譬如一周）在低峰时段进行数据一致性验证 注意点： * 要考虑异步复制的延迟，譬如可以考虑获取数据的timestamp，对最近1小时写入的数据不进行比较； * 可以考虑分布式并发执行，每个并发只比较一个partition，提升速度；

seperate key and large value in rocksdb

1

refer to WiscKey & HashKV

Appears for many times in unit test: ``` W2018-07-18 10:16:05.669 (1531880165669313326 617f) mimic.io-thrd.24959: io_getevents returns -4, you probably want to try on another machine:-( ```

potential secondary learn too slow to catch up with primary

3

In our test, we found that the potential secondary learn too slow: ``` log.201.txt:D12:14:41.839 (1470744881839169361 bc6c) replica5.replica_long3.080400030000001d: replica.learn:843:on_copy_remote_state_completed(): [email protected]:34805: on_copy_remote_state_completed[0000025200000002]: learnee = 10.108.187.19:34803, learn_duration = 3754 ms, apply checkpoint/log done,...

TODO in mutation::write_to(). refer to #499

shared log not flush to disk/ssd before return set result to client

9

in mutation_log_shared::append(), the shared log is not flushed (fsync on linux) before commit, which may cause data lost when machine restart under strong consistency semantics.

parallel execution of read and write may cause inconsistency

1

As in the following code: ``` MODULE_INIT_BEGIN(replication_type1) dsn_task_code_register("RPC_L2_CLIENT_READ", TASK_TYPE_RPC_REQUEST, TASK_PRIORITY_COMMON, THREAD_POOL_LOCAL_APP); dsn_task_code_register("RPC_L2_CLIENT_WRITE", TASK_TYPE_RPC_REQUEST, TASK_PRIORITY_LOW, THREAD_POOL_REPLICATION); dsn::register_layer2_framework< ::dsn::replication::replication_service_app>("replica", DSN_APP_MASK_FRAMEWORK); MODULE_INIT_END ``` on_client_read() is executed in LOCAL_APP thread pool, which is not...

``` // write-to/read-from mutation log file, for better performance // // TODO(qinzuoyan): the optimization is to marshall code as int but not string, // but this may cause problem, because...

hpc_logger lost last lines when core

2

when using hpc_logger, if coredump, the last (also the most important) lines will lost because not flushed.