KAFKA-17400: Added share fetch purgatory for delaying share fetch requests which cannot be completed because of reaching record lock partition limit
About
Introduced a share fetch purgatory on the broker which delays share fetch requests that cannot be completed instantaneously. Introduced 2 new classes -
- DelayedShareFetch - Contains logic to instantaneously complete or force complete a share fetch request on timeout.
- DelayedShareFetchKey - Contains the key which can be used to watch the entries within the share fetch purgatory.
- ShareFetchUtils - This utility class contains functionalities required for post-processing once the replica manager fetch is completed.
There are many scenarios which can cause a share fetch request to be delayed and multiple scenarios when a delayed share fetch can be attempted to be completed. In this PR, we are only targeting the case when record lock partition limit is reached, the ShareFetch should wait for up to MaxWaitMs for records to be released
Testing
The code has been tested with the help of unit tests.
Hi @junrao, @mumrah and @AndrewJSchofield, please re-review my code whenever you get a chance. Thanks.
Hi @junrao, I have addressed your comments. Please re-review whenever you can. Thanks!
The tests failing for JDK 21 and Scala 2.13 are unrelated to the PR. I ran the four of them locally and they all passed.