fetching block by range isn't 100% right...

[rolfyone]

when a peer requests more than 200 blocks in a range, we limit the range to 200.

This is probably usually ok, but there's a potential edge case, because we need to return at least 1 block.

It's possible that the first 200 blocks in the requested range don't exist, but there's a block further into the range that does, so we should really be searching past that 200 if required, so that we can return the first block we have, as we're supposed to return at least 1 block in the response if we have any.

ie. if we request 1024 blocks (max), and only block 1024 of that range is one that we have , we are expected to send back that block.

[seroze]

@rolfyone is this issue still open. can you point to the class where this is happening.

this seems like pagination hasn't been implemented properly with pageSize = 200 (max).

[ajsutton]

@rolfyone is this issue still open. can you point to the class where this is happening.

This is in BeaconBlocksByRangeMessageHandler.

this seems like pagination hasn't been implemented properly with pageSize = 200 (max).

Not exactly - it's more that we're limiting the size of the range we allow to be requested up front rather than limiting the number of slots we search after the first block. But finding that first block is hard because the request might have a skip value greater than 1 (ie asking for every 10th block). We can efficiently find the next entry in the database but could wind up having to scan huge sections of the db to find the first block.

Other clients just reject requests outright if they span too many slots. So my suggestion would be to track when we limit the number of blocks we'll search for and whether we found a block. If we hit the restricted limit without finding a block, return an error code rejecting the request. We wouldn't return an error if we found a block, or if we searched every requested slot, even if they were all empty.

So:

• Request slots 1-1000, and we find no blocks in the slots we check (1-201) - return an error response
• Request slots 1-201 and we find no blocks after checking them all - no error (and no response to confirm all slots were empty)
• Request slots 1-1000, and we do find a block - return the blocks we find with no error
• Request slots 1-201 and we find some blocks after checking them all - return the bocks we found with no error