pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[Enhancement] Support getting message index in `ManagedLedger` interface

Open liangyepianzhou opened this issue 7 months ago • 1 comments

Search before reporting

  • [x] I searched in the issues and found nothing similar.

Motivation

  1. PIP-415 supports getting a message ID by index. This capability relies on broker entry metadata to obtain the message index. However, custom message layer implementations (e.g., StreamNative's Ursa engine) may not utilize broker entry metadata for this purpose. see more details in https://github.com/apache/pulsar/pull/24222#issuecomment-2921951865
  2. The current implement of searching message index is inefficient, that needs to traverse all entries and check whether their indexes meet the requirements.

Solution

  1. Add a method findMessageIndexByPosition to the managerLedger interface.
  2. Record the first entry index in LedgerInfo—the manager ledger can directly compute the index based on the message ID, significantly improving API efficiency and avoiding read amplification.

Alternatives

Add a method

Anything else?

No response

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

liangyepianzhou avatar Jun 03 '25 03:06 liangyepianzhou