go-car icon indicating copy to clipboard operation
go-car copied to clipboard

feat: support .car files with no root nodes

Open mikeal opened this issue 4 years ago • 1 comments

Heya,

We’re using .car files as a block store format in a few new use cases and this bit here is giving us some trouble.

https://github.com/ipfs/go-car/blob/6bca8656ee37dcc2ed011117958f64427c2ef2ab/car.go#L109-L111

We’re storing very large files across multiple .car files. One of the .car files has a root node that describes all the chunks of the file and what other .car files the blocks can be retrieved at. But the .car files that just contain a bunch of raw blocks don’t have a logical root node.

Our options right now are to either add every block’s CID as a root, which seems unnecessary and a potential performance issue on read, or to create an unnecessary vector for all the links and put that in there as well, which is really just a bunch of unnecessary data we’re shoving into the file.

It would be nice if .car files could optionally not contain any root nodes at all.

mikeal avatar Dec 12 '19 00:12 mikeal