hostd
hostd copied to clipboard
Cache sector roots for proof building
As the amount of data stored by indexers on the network goes up, so will the number of integrity checks they perform. Right now each integrity check reads a full sector from disk to build the merkle proof for the returned data.
If every sector is checked every 2 weeks, a host storing ~5TB of data will have 1 sector (4MiB) read from disk per second. That is a lot of constant I/O for a not particularly large host. We should come up with a plan to reduce that wear on a host's disks.
A potentially quite efficient solution would be to hash 4KiB subtrees of the stored data. Leading to 32KiB of cached roots per 4MiB of stored data. An approximately 0.08% overhead. Going below 4KiB wouldn't give us any benefits in terms of I/O anyway since that's usually the atomic read/write size of modern disks.
That said, 0.08% is still almost 9GiB of overhead per 1TiB of stored data. So it won't be feasible to store this in RAM.