KAFKA-17774: Adding capability to handle max fetch records in Share Fetch (KIP-932)

[apoorvmittal10]

The PR adds capability to restrict the messages in Share Fetch. The max fetch records will be an additional way to limit the number of records sent from broker to client.

In Share Fetch, with min and mx bytes, there exists 3 problems:

1. The max.poll.records client config sends the max number of records defined to application but might have fetched extra becuase of higher max bytes. But the timeout for the sent records has started on the broker.
2. As the application processes records as per max.poll.records, hence those number of records are sent in every acknowledgement. This causes the cache data to be tracked per offset as the batch is broken.
3. The client has to sent the partial acknoledgment batch and cannot piggyback on fetch requests.

To handle the above scenario max fetch records has been added. Once this PR is merged and we define the right methodolgy then KIP will be updated to have max fetch records in share fetch RPC rather as broker config.

Committer Checklist (excluded from commit message)

• [ ] Verify design and implementation
• [ ] Verify test coverage and CI build status
• [ ] Verify documentation (including upgrade notes)

[apoorvmittal10]

@AndrewJSchofield Thanks for the review. I have address the comments.

[apoorvmittal10]

@adixitconfluent @junrao @AndrewJSchofield Can I please get the review again. I am rebasing the PR with upstream merge.

[apoorvmittal10]

@AndrewJSchofield @chia7712 @junrao Thanks for reviewing, appreciate efforts on the long PR. I have rebased the PR and addressed the comments, can I please get the review again.

[apoorvmittal10]

@AndrewJSchofield @chia7712 Please let me know if need to address anything in this PR or it's good to merge.

[apoorvmittal10]

@chia7712 I have addressed the comments. Thanks for the review.