ethersphere
feat: soc dispersed replica v2
Checklist
- [x] I have read the coding guide.
- [ ] My change requires a documentation update, and I have done it.
- [x] I have added tests to cover my changes.
- [x] I have filled out the description and linked the related issues.
Description
This PR is a continuation of the original PR https://github.com/ethersphere/bee/pull/5057.
Original description
Add support for dispersed replicas on Single Owner Chunks (SOCs) to improve data availability and retrieval reliability in the Swarm network.
SOC replicas are generated by allowing additional addresses to represent the same SOC. It is achieved by lighten the validation of SOCs which ignores the first byte of the address. This makes possible to saturate dispersed replicas evenly across the whole network since nodes arrange into neighborhoods based on address prefix. The addresses created in a way to iterate over all variations in the given depth of the redundancy level + 1 (e.g. level is 2, and the original address starts with 101, then it uploads SOCs with addresses same after the 3 first bits where the first 3 bit variations are 001, 011, (101 is not because the original address has it), 111 and 100 (flipping the last bit).
New changes
Fixes SOC dispersed replica functionality by setting explicit default redundancy levels and fixing implementation issues.
- Set default redundancy level to
PARANOIDfor handlers - Use
wg.Go()instead of manual goroutine management - Change redundancy level header to pointer type for proper optional handling
- Refactor feeds factory to use functional options pattern
- Add comprehensive tests for SOC replicas
- Introduce the iterator for iterating over replica addreses one by one for better concurrency management and as the original implementation had bugs
Open API Spec Version Changes (if applicable)
feed and soc replica PUT endpoints swarm-redundancy-level header: create and push dispersed replicas according to the passed level: MEDIUM 2, STRONG 4, INSANE 8, PARANOID 16.
feed and soc replica GET endpoints swarm-redundancy-level header: for calibrating how deeply dispersed replicas should be checked. By default it is zero.
Motivation and Context (Optional)
Related Issue (Optional)
Screenshots (if appropriate):
Hi @zelig. I would kindly ask you to review this PR for correctness and in general. It is a combined effort from multiple developers and we have tried to validate the behaviour through unit tests, but we would like to have your validation because of potential risks. Would you check also what should be default redundancy levels in api handlers if Swarm-Redundancy-Level is not set in the request?
