Proposal: make block stream yield blocks when requested

[0x009922]

How it works now

Currently, when Iroha has e.g. 1500 blocks and you connect to the block stream with height=1000, it will immediately emit (as WebSocket events) all existing 500 blocks, and then emit new blocks when ready.

Problem

It is not necessarily desirable. A client might need to take some time to process each incoming block (as is the case with the Explorer, for example). Keeping all received blocks from the stream in memory until then would be problematic when there are many of them.

Proposal

Introduce a mechanism to emit blocks only when the client tells it's ready to receive the next one.

Technically, I imagine BlockSubscriptionRequest to have an additional parameter - lazy:

struct BlockSubscriptionRequest {
  from_height: NonZero<u64>,
  lazy: bool
}

If client sets lazy: true, then Iroha would wait for the next invocation from the client:

enum BlockStreamClientMessage {
  /// Client is ready to receive the next block (applicable when `lazy: true`)
  Next
}

Downsides

Breaking ABI change.

Alternatives

None in my mind.

[s8sato]

In the current implementation, what happens to the behavior of the client or the explorer if they receive a large number of blocks at once?

[0x009922]

In the current implementation, what happens to the behavior of the client or the explorer if they receive a large number of blocks at once?

I suppose there will be a buffer overflow somewhere internally in the TCP stream or tungstenite. They will receive a lot of data from the socket, and will have to hold it until client consumes it.