ethersphere
Reserve stampindex timestamps not updated
Context
2.1.0 (and possibly earlier)
Summary
I am re-uploading existing data on an existing stamp to the swarm. The pushing stamper updates the existing stamp's timestamp in this case. But once the chunk is pushed, the storing node determines that the chunk/batch already exists in the reserve and does NOT update the timestamp in the stampindex. This causes old timestamps to continue to exist in the reserve. Also these obsolete (read: earlier) timestamps are subsequently pullsynced to other nodes.
Note: The stamp timestamp is critical for batch bucket/index replacements especially for mutable batches. But even for re-uploaded (and newly timestamped) chunks, it is important that all reserves have the SAME timestamp for a stamped chunk.
https://github.com/ethersphere/bee/blob/8c61408db7cd63227cfe161967e3b70f90b9fde3/pkg/postage/stamper.go#L49-L51
https://github.com/ethersphere/bee/blob/8c61408db7cd63227cfe161967e3b70f90b9fde3/pkg/storer/internal/reserve/reserve.go#L100-L106
Expected behavior
If the original stamp timestamp is updated, I would expect all reserve timestamps to be updated when the stamped chunk is pushed.
Actual behavior
Reserve timestamps are not updated if the stamped chunk is already in the reserve. See line numbers quoted in the possible solution.
Steps to reproduce
Since stamp information is well hidden in the swarm implementation, you really can't see this happening without adding additional stamp and reserve timestamp logging. You can find my implementation of this logging in the following files of my hacks branch. Of course, other files were also modified to enable the logs. pkg/postage/stamper.go pkg/storer/internal/reserve/reserve.go https://github.com/ldeffenb/bee/tree/2.1.0-hacks
Possible solution
It's going to kill performance, but even if the reserve already Has the chunk/batch, the stampindex timestamp should still be verified and/or updated as necessary. https://github.com/ethersphere/bee/blob/8c61408db7cd63227cfe161967e3b70f90b9fde3/pkg/storer/internal/reserve/reserve.go#L100 https://github.com/ethersphere/bee/blob/8c61408db7cd63227cfe161967e3b70f90b9fde3/pkg/storer/internal/reserve/reserve.go#L116 But please do NOT update the store if the new timestamp matches the original! And also don't log it as an error either! https://github.com/ethersphere/bee/blob/8c61408db7cd63227cfe161967e3b70f90b9fde3/pkg/storer/internal/reserve/reserve.go#L123
It's worse than just the timestamp. Pushsync into the reserve will also not properly store a re-stamped chunk that happens to have been re-stamped into a different INDEX of the batch's bucket because the chunk/batch is already in the reserve. This is a critical flaw in mutable stamps.
And worse, since many chunks exist in the swarm whose stamps were forgotten in the great batch purge, those chunks CANNOT be reliably re-stamped with the original stamp because they may end up at different indices which will not be properly propagated to nodes still holding the original stamped chunks.
