pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[improve][client] Make seek wait for reconnect

Open nodece opened this issue 3 years ago • 1 comments

Motivation

When doing seek operation, the client needs to seed a seek command to the broker. When one seeks is successful, the broker disconnects the client, and then the client reconnects and resubscribes to the broker based on the seek position.

Right now, the consumer responds to the caller after sending a seek command succeeds, this should be incorrect behavior. The right behavior should be to wait for reconnect and resubscribe, then respond to the caller.

Modifications

Based on the global CompletableFuture and seekMessageId check the seek operation to wait for reconnect and resubscribe.

When reconnecting is done, make the CompletableFuture to complete, when resubscribing is done, cleanup the seekMessageId.

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/nodece/pulsar/pull/9

nodece avatar Oct 26 '22 03:10 nodece

Codecov Report

Merging #18202 (68ba899) into master (6c65ca0) will increase coverage by 5.44%. The diff coverage is 35.90%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #18202      +/-   ##
============================================
+ Coverage     34.91%   40.35%   +5.44%     
- Complexity     5707     8685    +2978     
============================================
  Files           607      687      +80     
  Lines         53396    67396   +14000     
  Branches       5712     7219    +1507     
============================================
+ Hits          18644    27200    +8556     
- Misses        32119    37161    +5042     
- Partials       2633     3035     +402     
Flag Coverage Δ
unittests 40.35% <35.90%> (+5.44%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...org/apache/bookkeeper/mledger/LedgerOffloader.java 0.00% <ø> (ø)
...che/bookkeeper/mledger/LedgerOffloaderFactory.java 0.00% <ø> (ø)
...pache/bookkeeper/mledger/LedgerOffloaderStats.java 0.00% <ø> (ø)
...ookkeeper/mledger/LedgerOffloaderStatsDisable.java 0.00% <ø> (ø)
...a/org/apache/bookkeeper/mledger/ManagedCursor.java 85.71% <ø> (ø)
...che/bookkeeper/mledger/ManagedLedgerException.java 41.17% <ø> (ø)
...bookkeeper/mledger/ManagedLedgerFactoryConfig.java 86.66% <ø> (ø)
...g/apache/bookkeeper/mledger/ManagedLedgerInfo.java 22.22% <ø> (ø)
...he/bookkeeper/mledger/OffloadedLedgerMetadata.java 0.00% <ø> (ø)
...ava/org/apache/bookkeeper/mledger/ScanOutcome.java 0.00% <ø> (ø)
... and 741 more

codecov-commenter avatar Nov 02 '22 05:11 codecov-commenter

@nodece Do you think that this PR addresses #10671 ?

lhotari avatar Nov 23 '22 07:11 lhotari

@nodece Do you think that this PR addresses #10671 ?

I think this PR can fix race condition of seeking.

nodece avatar Nov 23 '22 07:11 nodece

is there any relationship to #16757?

lhotari avatar Nov 23 '22 16:11 lhotari

is there any relationship to #16757?

Right, but this PR fixes some issues on the client side.

nodece avatar Nov 24 '22 02:11 nodece

I think we should reconsider seek. This PR only solves some problems, but it does not solve all seek and reconnect concurrency problems.

Do you have any suggestions?

nodece avatar Dec 26 '22 06:12 nodece

I have no idea about that. [PIP-194] https://lists.apache.org/thread/97o9t4ltkds5pfq41l9xbbd31t41qm8w discuss how to solve this problem.

congbobo184 avatar Dec 28 '22 06:12 congbobo184

I have no idea about that. [PIP-194] https://lists.apache.org/thread/97o9t4ltkds5pfq41l9xbbd31t41qm8w discuss how to solve this problem.

I think this is a different issue, I just fix the concurrency problems of seeking on the client side.

nodece avatar Dec 28 '22 07:12 nodece

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Jan 28 '23 02:01 github-actions[bot]

No reviewer, so close this PR.

nodece avatar Mar 25 '23 17:03 nodece