pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][offload]fix get error ledgerhandle when ledger trim

Open ethqunzhong opened this issue 6 months ago • 2 comments

relate issue: https://github.com/apache/pulsar/issues/21737

Motivation

fix get error ledgerhandle when ledger trimed case NoSuchLedgerExitsException

Modifications

cursor position at trimming-ledger which to be deleted from Bookkeeper.After the ledger deleted due to archive the ManagedLedgerOffloadDeletionLagInMillis, readentries should use new readhanle implement by OFFLOADreadhandle, in current implementation, get readhandle generate from Bookkeeper, use it to would cause read will cause NoSuchLedgerException.

Regarding the addition of ledgers to offloadedLedgersToDelete, there are two scenarios to consider:

  1. BOOKKEEPER_FIRST: If a ledger is currently being read from, it should not be added to offloadedLedgersToDelete during the current trim process to prevent read exceptions from BookKeeper. If the ledger is not being read from, the readHandle from BookKeeper should be invalidated. Subsequent reads of this ledger will then reacquire the readHandle from tiered storage.

  2. TIERED_STORAGE_FIRST: If isOffloadedNeedsDelete is set to true, continue adding to offloadedLedgersToDelete because the current reading method utilizes tiered storage. Deleting a ledger from BookKeeper will not impact the ongoing process. After the ledger is deleted from BookKeeper, it is unnecessary to invalidate the readHandle because if the current LedgerHandle in offloadedLedgersToDelete is reading data from tiered storage, invalidating the readHandle could result in a NullPointerException (NPE).

Verifying this change

  • [x] Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [ ] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [ ] The REST endpoints
  • [ ] The admin CLI options
  • [ ] The metrics
  • [ ] Anything that affects deployment

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/ethqunzhong/pulsar/pull/10

ethqunzhong avatar Dec 16 '23 12:12 ethqunzhong