pulsar
pulsar copied to clipboard
[fix][offload]fix get error ledgerhandle when ledger trim
relate issue: https://github.com/apache/pulsar/issues/21737
Motivation
fix get error ledgerhandle
when ledger trimed case NoSuchLedgerExitsException
Modifications
cursor position at trimming-ledger
which to be deleted from Bookkeeper.After the ledger deleted due to archive the ManagedLedgerOffloadDeletionLagInMillis,
readentries should use new readhanle implement by OFFLOADreadhandle
, in current implementation, get readhandle
generate from Bookkeeper, use it to would cause read will cause NoSuchLedgerException
.
Regarding the addition of ledgers to offloadedLedgersToDelete
, there are two scenarios to consider:
-
BOOKKEEPER_FIRST: If a ledger is currently being read from, it should not be added to
offloadedLedgersToDelete
during the current trim process to prevent read exceptions from BookKeeper. If the ledger is not being read from, thereadHandle
from BookKeeper should be invalidated. Subsequent reads of this ledger will then reacquire thereadHandle
from tiered storage. -
TIERED_STORAGE_FIRST: If
isOffloadedNeedsDelete
is set to true, continue adding tooffloadedLedgersToDelete
because the current reading method utilizes tiered storage. Deleting a ledger from BookKeeper will not impact the ongoing process. After the ledger is deleted from BookKeeper, it is unnecessary to invalidate thereadHandle
because if the currentLedgerHandle
inoffloadedLedgersToDelete
is reading data from tiered storage, invalidating the readHandle could result in aNullPointerException
(NPE).
Verifying this change
- [x] Make sure that the change passes the CI checks.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
Documentation
- [ ]
doc
- [ ]
doc-required
- [x]
doc-not-needed
- [ ]
doc-complete
Matching PR in forked repository
PR in forked repository: https://github.com/ethqunzhong/pulsar/pull/10
@ethqunzhong Please add the following content to your PR description and select a checkbox:
- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
@ethqunzhong please rebase this PR