rust-rocksdb icon indicating copy to clipboard operation
rust-rocksdb copied to clipboard

use bindgen to auto generate ffi rust code

Open fredchenbj opened this issue 6 years ago • 7 comments

refer to https://github.com/tikv/rust-rocksdb/issues/381.

just use old c.h and bindgen to gen ffi's rust code. and fix to pass cargo build.

this is step 1, later replace with rocksdb's c.h to gen code.

Signed-off-by: fredchenbj [email protected]

fredchenbj avatar Dec 03 '19 04:12 fredchenbj

Please fix the conflicts

Connor1996 avatar Dec 04 '19 02:12 Connor1996

Please fix the conflicts

@Connor1996 I have fixed the conficts.

fredchenbj avatar Dec 04 '19 02:12 fredchenbj

@fredchenbj awesome changes! Some general questions:

  1. Can you summarize what's changed in this PR? Seems mostly typing changes. Why are they needed?
  2. The PR seems not including the generated code. Should we checkin the generated code or not?
  3. What's the overall plan for migration?

Thanks.

yiwu-arbug avatar Dec 04 '19 04:12 yiwu-arbug

It introduces clang dependency, which can be a pain to fix. You can learn how to avoid it from tikv/grpc-rs.

BusyJay avatar Dec 04 '19 05:12 BusyJay

@BusyJay You mean pre-generate binding? Yeah, that's what we plan to do. For easy reviewing, we plan to do it step by step.

Connor1996 avatar Dec 04 '19 06:12 Connor1996

@fredchenbj awesome changes! Some general questions:

  1. Can you summarize what's changed in this PR? Seems mostly typing changes. Why are they needed?
  2. The PR seems not including the generated code. Should we checkin the generated code or not?
  3. What's the overall plan for migration?

Thanks.

@yiwu-arbug Thanks for your questions.

for 3, the migration plan has three steps: 1) use the old crocksdb's c.h and bindgen to generate ffi rust code, and make it work for all dependency code; 2) replace with rocksdb's c.h, and make all code in c.h conform to rocksdb's style, here we would only include rocksdb's c.h and not change it at all; 3) try to use rocksdb cpp api to generate ffi rust code. This pr is only step 1.

for 1, this pr has mainly three changes: 1) change crocksdb's c.h and c.cc to conform to bindgen's requirement; 2) use bindgen to generate ffi rust code; 3) change code in rust-rocksdb, including many type changes, to make those code could call the code geneated by bindgen.

for 2, the generated code is in target/debug/build/librocksdb_sys-f21a7cfdab79a481/out/bindings.rs, which is auto generated by cargo build. IMHO, we don't need checkin it.

fredchenbj avatar Dec 04 '19 09:12 fredchenbj

@fredchenbj for 3, can you give more details for step 1)? Does the generated code provide difference interface than the existing rust API? If so, how much work is it to update tikv? If that work is huge, is there way to split the work into small bits?

Also for 3, if we will trying generating code from c++ directly, how useful step 2) would be?

Currently the wrapper doesn't handle Status return from rocksdb well. It convert the struct to string and miss the typing. I hope with step 3) we can change it to convert to rust Result.

yiwu-arbug avatar Dec 06 '19 21:12 yiwu-arbug