datamon
datamon copied to clipboard
core: unpack highly sentitive to the choice of bundleEntriesPerFile
Trying to reuse pkg/core/bundle_test.go to be used in other tests, I discovered that if a bundle has been created with a given "bundleEntriesPerFile", it cannot be unpacked later on with a different setting.
E.g. the mock creates, say, 4 files split in 2 index files with 2 entries. When calling "Mount" which uses the default of 1000 files per index, this breaks.
This means that we cannot freely change this number as it will break all previously created bundles.
there are some conventions over configuration scattered throughout the application.
another constant that exists, where changing it could break source-data compatability, is the blob size.
needless to say, pack and unpack are sensitive to the choice of hashing algorithm as well, we don't abstract out the fact that we use BLAKE rather than SHA, also for instance.
not drawing conclusions here about what needs to be changed and in what order. just saying there are lots of things that could be additionally abstracted, not just this constant.