pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix] [broker] make rest api PersistentTopic#getLastMessageId consistent with binary api ServerCnx#handleGetLastMessageId

Open thetumbled opened this issue 1 year ago • 1 comments

Fixes https://github.com/apache/pulsar/issues/21968

Motivation

The implementation of org.apache.pulsar.broker.service.persistent.PersistentTopic#getLastMessageId is not consistent with org.apache.pulsar.broker.service.ServerCnx#handleGetLastMessageId, for example, ServerCnx#handleGetLastMessageId may get last message id from the compacted topic, while PersistentTopic#getLastMessageId don't.

~~And flink-pulsar-connector also rely on the rest api PersistentTopic#getLastMessageId, we have better fix this problem.~~ Older version of flink-pulsar-connector use it, latest version change to call consumer.getLastMessageId();.

Modifications

Refactor PersistentTopic#getLastMessageId to be simillar with ServerCnx#handleGetLastMessageId.

Verifying this change

  • [x] Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change added tests and can be verified as follows:

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [ ] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [x] The REST endpoints
  • [ ] The admin CLI options
  • [ ] The metrics
  • [ ] Anything that affects deployment

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/thetumbled/pulsar/pull/37

thetumbled avatar Jan 25 '24 09:01 thetumbled

As #21969 could retrieve the last message id from the compacted topic when consumer is in read compact mode, i wonder should we add a param readCompact to simulate this behavior, or we can just let these two api inconsistent? @coderzc @Technoboy- @codelipenghui @liangyepianzhou @poorbarcode

thetumbled avatar Jan 31 '24 09:01 thetumbled