lotus
lotus copied to clipboard
fix: fr32: Make UnpadReader be a correct Reader
Related Issues
Fixes https://github.com/filecoin-project/lotus/issues/9324
Proposed Changes
The fr32 unpad reader currently requires that the output is power-of-two-unpadded size, which is an extremely sketchy assumption and seems to cause lots of pain in retrievals.
There is already quite a lot of code in piece-provider which ensures that reads are initialized at correct offsets (multiple of 127-unpadded-bytes), but the read sizes are only ensured by the fact that the output reader is wrapped in a bufio.Reader, which won't always buffer writes with the required size, leading to hard to debug panics.
This PR aims to make the UnpadReader behave more like any normal reader, removing a giant footgun in the retrieval code.
Additional Info
I realize that this code should probably eventually be external to lotus, but right now both boost and curio depend on this for decoding fr32 data. Should be extracted but let's fix the panics first.
Checklist
Before you mark the PR ready for review, please make sure that:
- [ ] Commits have a clear commit message.
- [ ] PR title conforms with contribution conventions
- [ ] Update CHANGELOG.md or signal that this change does not need it per contribution conventions
- [ ] New features have usage guidelines and / or documentation updates in
- [ ] Tests exist for new functionality or change in behavior
- [ ] CI is green