hypercore-protocol
What are the storage limits for hyperdrive?
This is originally posted here, but since this is the next version preview going on, thought of adding here so that this can be discussed and can possibly contribute to the design / architecture.
- (Assuming we have enough hard-disk space) Can hyperdrive be used to store large files (say 100K+ files, each 100GB) ?
When using a local file system, it looks like all the hyperdrive block data is stored into a single file (/data) in the storage. As such that will have problems with the OS file size limits.
What happens when the hyperdrive content becomes large? Will the hyperdrive (or hyperspace / hypercore) continue to append the blocks to the same underlying storage data file or will it spill them into different data files?
Where can one find more information on how the file system is organized underlying for the blobs of Hyperdrive?
It continues to append to that file yes, but the file impl is flexible so close to trivial to make it use multiple if you wanted to (see random-access-storage)
