kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-18681 Created GetReplicaLogInfo* RPCs

Open josefk31 opened this issue 8 months ago • 3 comments

Creates GetReplicaLogInfoRequest and GetReplicaLogInfoResponse RPCs Information returned by these brokers will be used to aid unclean-recovery by selecting longest logs.

RPCs are cluster actions, meaning they may only be queried by controllers to other brokers.

Delete this text and replace it with a detailed description of your change. The PR title and body will become the squashed commit message.

See KIP-966 for more information on this RPC.

josefk31 avatar May 08 '25 20:05 josefk31

We need a boolean to indicate that there is more data that we were unable to send due to length restrictions

cmccabe avatar May 15 '25 20:05 cmccabe

We need a boolean to indicate that there is more data that we were unable to send due to length restrictions

I wonder if it is good enough to indicate the limit is reached when the partition is missing from the response? Especially, we use an explicit partition list in the request. @cmccabe

CalvinConfluent avatar May 19 '25 15:05 CalvinConfluent

We need a boolean to indicate that there is more data that we were unable to send due to length restrictions

I wonder if it is good enough to indicate the limit is reached when the partition is missing from the response? Especially, we use an explicit partition list in the request. @cmccabe

@CalvinConfluent I think we're dealing with the following tradeoffs:

  1. We want to save space on RPCs
  2. We want to reduce processing time on the brokers
  3. RPC should be predictable

It's more obvious what the RPC is doing if we return an error code for each partition above the limit. That being said; I think it does not address (1) completely. I think that the Truncate is balancing (1) and (2) at the expense of (3) being a bit less clear. It's still fairly straightforward that we return the first 1k requested (in order) and set Truncate to true and requires a consumer of the API to read the API definition. But since this is a cluster-action I feel that the confusion is fair to impose?

josefk31 avatar May 21 '25 16:05 josefk31