valkey icon indicating copy to clipboard operation
valkey copied to clipboard

Skip CRC64 checksumming when doing diskless replication

Open madolson opened this issue 1 year ago • 1 comments

When we are not writing to disk (either on the primary or replica) there isn't a lot of sense in doing CRC64 checksumming during fullsync. We do checking summing at the TCP layer already. At AWS we've seen it can take up to 15% overhead for not much of a benefit. We should look into a mechanism so that a replica can indicate that it will load something directly into memory, since today we only indicate we support the EOF marker. The replica might still choose to save the RDB file to disk unless repl-diskless-load is enabled. With this mechanism in place, we should be able to skip the checksum for replication, while still using it for RDB.

madolson avatar Oct 05 '24 01:10 madolson

Hi @madolson , Please assign this to me, I will work on it. Thanks!

Shivshankar-Reddy avatar Oct 07 '24 17:10 Shivshankar-Reddy