tikv
Support for ChecksumGeneratorFactory
It would be nice to have a trait for implementing customized checksum generator factories as described in https://github.com/facebook/rocksdb/wiki/Full-File-Checksum-and-Checksum-Handoff#how-to-use-full-file-checksum
The API is here https://github.com/facebook/rocksdb/blob/main/include/rocksdb/file_checksum.h
I'm not sure how this could be implemented but I would be glad to help.
Yes it would be nice. But before we start working on the details, I want to be sure that you understand the current status of this crate. Right now, the master branch is linked with TiKV's fork of RocksDB 6.29 (https://github.com/tikv/rocksdb). And we don't have a clear plan to update it yet. In fact, we are lowkey working on a new binding crate tirocks designed to replace this one. When that happens, this crate will likely be archived.
The version isn't a problem as 6.29 supports full file checksum. But in that case, do you recomend using tirocks instead?
do you recomend using tirocks instead?
tirocks is not production ready yet. But it would be fairly easy to port full checksum then because tirocks has a superior binding generation workflow.
The version isn't a problem as 6.29 supports full file checksum.
Great. To port full checksum in rust-rocksdb, you can start with the existing code of compaction filter. They have a very similar structure.
My recent commit refactored the compaction filter interface (https://github.com/tikv/rust-rocksdb/commit/9e4678857e5b4c738e95c7ee1a35ee962264f4e9). You can pretty much see how a binding is implemented: (1) declare ffi in c.h (2) implement ffi in c.cc (3) implement rust wrapper of raw C pointer. For structures involving callbacks (such as compaction filter or checksum generator), we need to pass Rust function as C function pointer, and Rust struct as C raw pointer. You can see it in action with CompactionFilterContext or CompactionFilterFactoryProxy.