celestiaorg
fix(store): adding_storage_sharding
This should have a test that verifies the expected FS structure on disk is created.
Additionally, is this a breaking change? I would assume yes since we are changing how data is being written/read from disk, so existing data would not be found? To test/verify that, we should write a test that starts with data written in the previous structure, and then tries to read it.
There is an existing test that checks for the expected FS structure, but it needs to be fixed, so the tests are failing.
Also, while working on this during onsite, we realized that sharding for height indicies wont work with hardlinks (as it currently stands) and we should discuss how to proceed here.
Its either we migrate over symlinks or solve sharding on FS level, as discussed in #4100
@sysrex, if you want to continue here, the next step is to migrate from hardlinks to symlinks. Would be also great to analyze potential performance implication caused by this transition
@sysrex, if you want to continue here, the next step is to migrate from hardlinks to symlinks. Would be also great to analyze potential performance implication caused by this transition
on it
https://github.com/celestiaorg/celestia-node/compare/v0.21.9-mammoth-v0.0.15...v0.21.9-mammoth-v0.0.16
