lotus icon indicating copy to clipboard operation
lotus copied to clipboard

fix: fr32: Make UnpadReader be a correct Reader

Open magik6k opened this issue 5 months ago • 2 comments

Related Issues

Fixes https://github.com/filecoin-project/lotus/issues/9324

Proposed Changes

The fr32 unpad reader currently requires that the output is power-of-two-unpadded size, which is an extremely sketchy assumption and seems to cause lots of pain in retrievals.

There is already quite a lot of code in piece-provider which ensures that reads are initialized at correct offsets (multiple of 127-unpadded-bytes), but the read sizes are only ensured by the fact that the output reader is wrapped in a bufio.Reader, which won't always buffer writes with the required size, leading to hard to debug panics.

This PR aims to make the UnpadReader behave more like any normal reader, removing a giant footgun in the retrieval code.

Additional Info

I realize that this code should probably eventually be external to lotus, but right now both boost and curio depend on this for decoding fr32 data. Should be extracted but let's fix the panics first.

Checklist

Before you mark the PR ready for review, please make sure that:

magik6k avatar Sep 19 '24 12:09 magik6k