go-car icon indicating copy to clipboard operation
go-car copied to clipboard

Indexes don't take advantage of performant structure

Open willscott opened this issue 2 years ago • 3 comments

The common form of Index introduced in Car V2 is to lay out the CIDs in the car in a sorted table that can be quickly walked through with a binary search.

We should attempt to optimize not fully copying the index into memory when loading a car, and just doing spot reads from the index. This can for large cars avoid a copy of hundreds of megs per individual block read.

willscott avatar Jan 20 '22 08:01 willscott