bookkeeper icon indicating copy to clipboard operation
bookkeeper copied to clipboard

[fix] Ledger Replication worker should throttle on verrification reads

Open vraulji567 opened this issue 2 months ago • 2 comments

Motivation

Replication worker fails to replicate ledgers with thousands of entries when auditorLedgerVerificationPercentage is set to 100. The verification process makes too many read requests at once exceeding maxPendingReadRequestsPerThread and fails with Too many requests to the same Bookie while reading. The verification process should not make too many simultaneous reads and throttle the reads based on inFlightReadEntryNumInLedgerChecker.

Changes

Pass the config to the verification process(LedgerChecker) while initializing it as part of the Replication worker to throttle the number of simultaneous reads to the bookie preventing Too many requests to the same Bookie while reading error and replication of large ledgers with 100% verification possible.

Added a test to verify the throttling logic.

vraulji567 avatar Nov 04 '25 22:11 vraulji567