lakeFS
lakeFS copied to clipboard
Make LakeFSInputFile (metadata reader) JOINable in (almost) linear time
Speed up metadata client JOIN operations by telling the driver how to JOIN metaranges in linear time. We will use this to transform garbage collector from a quadratic process to an (almost) linear process (O(n log n), in practice, due to cost of seeking).
Metaranges are sorted, so this should be possible by telling LakeFSInputFile to return a ComposableRecordReader.