leveldb icon indicating copy to clipboard operation
leveldb copied to clipboard

Make DBImpl designed for extensibility

Open subes opened this issue 4 years ago • 2 comments

In ezdb-leveldb I optimized away some unnecessary byte[] copies by exposing Slice methods directly in DBImpl and making the DBComparator zero-copy. This almost doubled the performance for iteration from ~1000/ms to ~2000/ms. So unnecessary copying seems to be a more significant performance hog. Can we incorporate the extensions to the three internal leveldb classes I made for this here (https://github.com/criccomini/ezdb/tree/master/ezdb-leveldb/src/main/java/org/iq80/leveldb)? I guess we could make the changes backwards compatible by just allowing subclasses of DBImpl to override implementations for UserComparator via a protected factory method. The methods that operate on Slices could be added from the outside. I prepared in "ExtensibleDBImpl" a version that is designed for extensibility (though more private methods could be made protected). I only modified all fields being protected and making the methods protected that I need for my extensions.

subes avatar Sep 19 '21 14:09 subes

At time being, the leveldb-api is designed to be compatible with JNI implementation and Java. This has two downsides; the java implementation has difficulties evolve/add new features not present in the original CPP implementation and require the adaptation of byte[] based API to an internal Slice based one.

I don't think DB should be extendable, but rather implement a new, more efficient, Java API based directly on Slice and proper Options. We can always support this old 'byte[]' based API with an adaptation layer.

pcmind avatar Sep 21 '21 10:09 pcmind

I will definitely integrate anything that you can provide ^^ Until then I will stick to my own copy of the class.

subes avatar Sep 21 '21 23:09 subes