HomeStore icon indicating copy to clipboard operation
HomeStore copied to clipboard

[DNM] add uint64 checksum log for blk data

Open JacksonYao287 opened this issue 2 months ago • 4 comments

this PR add uint64_t checksum for push_data/on_push_data_received and on_fetch_data_received/handle_fetch_data_response, so that we can directly check whether data is corrupted during network transmission from the log.

next step, we need to drop the received data if it is found corrupted.

JacksonYao287 avatar Sep 15 '25 06:09 JacksonYao287

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

:x: Patch coverage is 0% with 32 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 55.24%. Comparing base (1a0cef8) to head (dca9acf). :warning: Report is 264 commits behind head on master.

Files with missing lines Patch % Lines
src/lib/replication/repl_dev/raft_repl_dev.cpp 0.00% 32 Missing :warning:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #804      +/-   ##
==========================================
- Coverage   56.51%   55.24%   -1.28%     
==========================================
  Files         108      110       +2     
  Lines       10300    13281    +2981     
  Branches     1402     1935     +533     
==========================================
+ Hits         5821     7337    +1516     
- Misses       3894     5104    +1210     
- Partials      585      840     +255     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Sep 15 '25 07:09 codecov-commenter

[09/15/25 05:00:00-07:00] [D] [test_raft_repl_dev_replica_1] [1169944] [raft_repl_dev.cpp:1398:handle_fetch_data_response] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data received in handle_fetch_data_response for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=294958 count=2 chunk=4},] ] state=[BLK_ALLOCATED | DATA_RECEIVED | ]] is 13456295685418586948

[09/15/25 05:00:00-07:00] [D] [test_raft_repl_dev_replica_0] [1169527] [raft_repl_dev.cpp:1354:operator()] [traceID=n/a] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data in on_fetch_data_received for dsn=47 is 13456295685418586948

[09/15/25 04:59:45-07:00] [D] [test_raft_repl_dev_replica_2] [1169638] [raft_repl_dev.cpp:960:on_push_data_received] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data received in on_push_data_received for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=163922 count=2 chunk=3},] ] state=[BLK_ALLOCATED | DATA_RECEIVED | ]] is 13456295685418586948

[09/15/25 04:59:45-07:00] [D] [test_raft_repl_dev_replica_0] [1169464] [raft_repl_dev.cpp:856:push_data_to_all_followers] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data pushed for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=294997 count=2 chunk=3},] ] state=[BLK_ALLOCATED | ]] is 13456295685418586948

JacksonYao287 avatar Sep 16 '25 00:09 JacksonYao287

above is the produced log for push_data_to_follower/ on_push_data_received , on_fetch_data_received/ handle_fetch_data_reponse . all the member will calculate the checksum of the sent/received data and put it in the log.

JacksonYao287 avatar Sep 16 '25 00:09 JacksonYao287

DNM before we have upgrade compatibility implemented

xiaoxichen avatar Sep 24 '25 12:09 xiaoxichen