pmdk icon indicating copy to clipboard operation
pmdk copied to clipboard

obj: ensure zones are reclaimed prior to free

Open pbalcer opened this issue 2 years ago • 5 comments

This patch fixes a bug where pfree(), combined with reservations, prior to any allocs could have led to overlapping allocations - which ultimately was causing corrupted heap and incorrect statistics.

This problem is caused by lazy heap runtime state reclamation. Heap runtime state is rebuilt lazily whenever required to serve allocation requests. Deallocations (free) simply updates persistent metadata and, in case of huge allocations, inserts the freed chunk into a container of free chunks. On reclaim, all free chunks not already deallocated are inserted into a freelist.

This would have been fine, but libpmemobj's allocator enables software to reserve a chunk, removing it from the heap runtime state, without updating the persistent on-media layout. This means that software can deallocate a chunk, reserve that same chunk, allocate something normally - triggering heap zone reclamation, and then it can finally publish (actually persistently allocate) that reserved chunk. This can lead to the same chunk being potentially allocated twice...

This patch fixes this problem by ensuring that object's zone is fully processed and reclaimed prior to deallocation.

Reported-by: @jolivier23


This change is Reviewable

pbalcer avatar Jul 18 '22 14:07 pbalcer

please change the target branch to stable-1.X

lplewa avatar Jul 18 '22 14:07 lplewa

I think the name at the bottom got truncated :-)

jolivier23 avatar Jul 18 '22 15:07 jolivier23

Codecov Report

Merging #5470 (e21d3e2) into stable-1.12 (b2b1563) will increase coverage by 0.08%. The diff coverage is 80.00%.

@@               Coverage Diff               @@
##           stable-1.12    #5470      +/-   ##
===============================================
+ Coverage        72.21%   72.30%   +0.08%     
===============================================
  Files              193      193              
  Lines            30334    30535     +201     
===============================================
+ Hits             21906    22078     +172     
- Misses            8428     8457      +29     

codecov[bot] avatar Jul 19 '22 00:07 codecov[bot]

Rebased to 1.12

pbalcer avatar Jul 20 '22 10:07 pbalcer

was this issue introduced in 1.12 or in the older version? If in the older version, please rebase to the correct stable branch.

Reviewable status: 5 of 11 files reviewed, 1 unresolved discussion (waiting on @DamianDuy and @jolivier23)

It's definitely an issue in 1.11.0 as well. I can't speak to prior versions but I suspect it's always been an issue since we've been using PMDK.

jolivier23 avatar Aug 05 '22 15:08 jolivier23

Merged in #5489 and backported to 1.12.

pbalcer avatar Aug 17 '22 10:08 pbalcer