bookkeeper icon indicating copy to clipboard operation
bookkeeper copied to clipboard

[improve] Prevent memory copy and heap memory allocation when read entry

Open dao-jun opened this issue 1 year ago • 3 comments

Motivation

Prevent memory copy when read entry.

Changes

(Describe: what changes you have made)

Master Issue: #


In order to uphold a high standard for quality for code contributions, Apache BookKeeper runs various precommit checks for pull requests. A pull request can only be merged when it passes precommit checks.


Be sure to do all of the following to help us incorporate your contribution quickly and easily:

If this PR is a BookKeeper Proposal (BP):

  • [ ] Make sure the PR title is formatted like: <BP-#>: Description of bookkeeper proposal e.g. BP-1: 64 bits ledger is support
  • [ ] Attach the master issue link in the description of this PR.
  • [ ] Attach the google doc link if the BP is written in Google Doc.

Otherwise:

  • [ ] Make sure the PR title is formatted like: <Issue #>: Description of pull request e.g. Issue 123: Description ...
  • [ ] Make sure tests pass via mvn clean apache-rat:check install spotbugs:check.
  • [ ] Replace <Issue #> in the title with the actual Issue number.

dao-jun avatar Apr 02 '24 05:04 dao-jun

@lhotari @eolivelli @hangc0276 @zymap Could you please take a look?

dao-jun avatar Apr 05 '24 07:04 dao-jun

I am not familiar with netty reference count. I have two questions.

  1. why we need onSendResponseFinished? Can we just change to UnsafeByteOperations.unsafeWrap(entryBody.nioBuffer())? It may leak memory?
  2. I think extract a class field is a little hard to maintenance, can we still pass it by method?

hezhangjian avatar Apr 10 '24 09:04 hezhangjian

I am not familiar with netty reference count. I have two questions.

  1. why we need onSendResponseFinished? Can we just change to UnsafeByteOperations.unsafeWrap(entryBody.nioBuffer())? It may leak memory?
  2. I think extract a class field is a little hard to maintenance, can we still pass it by method?

@shoothzj

  1. UnsafeByteOperations is protobuf‘s API, it will not release the memory, so we have to release it manually.
  2. Pass body by method will lead to more changes, this is the most simple way.

dao-jun avatar Apr 10 '24 10:04 dao-jun