billy
billy copied to clipboard
shelf: use files capped to certain size
This PR adds 'cappedFile' support. A cappedFile behaves like a regular os.File
, but it actually maps to a set of files, each capped to a max size.
By swapping out the regular files to cappedFile, as backing for the shelves, billy will can be made to respect max file sizes in filesystem (e.g 4GB in fat32).
The cappedFile is not concurrency-safe for spread-out read/writes. That is, if the data to be read crosses file boundaries, then simultaneous read and write may cause data to be corrupted.
However, this can be easily avoided on the upper layer: the shelf can just ensure that the cappedFile limit is a multiple of the shelf size. So instead of using 2 * 1024 * 1024= 2097152
, for shelf-size 10
, it could use 2097150
. If it did, then the write-offsets (2097140
, 2097150
,2097160
) all occur so no writes crosses file boundaries.