leapfrog icon indicating copy to clipboard operation
leapfrog copied to clipboard

Leapfrog insert() keeps spinning in endless probe loop

Open velvia opened this issue 1 year ago • 0 comments

Hi there, I have a reproducible case of where Leapfrog map is endlessly going in a probing loop during inserts. MacBook Pro 2021, M1 Pro, OSX 12.1

Repo and commit is here (branch is try-leapfrog): https://github.com/velvia/ying-profiler/commit/cec84d0b42360d87ddf2a95b35a3da27555e88f4

To reproduce, run cargo test stress on nightly. The test will seem to hang. Use gdb/lldb to attach to the test process. You will find a stack trace like this:

* thread #3
  * frame #0: 0x0000000104481338 allocation_tests-de2cce5ea52d024e`leapfrog::leapmap::get_cell::ha8ca75b7371b4c44(buckets=&[leapfrog::leapmap::Bucket<u64, ying_profiler::AllocInfo>] @ 0x000000016bf31660, index=5117530326985502155, size_mask=5371614424) at leapmap.rs:1281
    frame #1: 0x000000010447d4ac allocation_tests-de2cce5ea52d024e`leapfrog::leapmap::LeapMap$LT$K$C$V$C$H$C$A$GT$::insert_or_find::hec008331e7fccd14(hash=5117530326985502029, key=0x000000016bf31a10, value=(__0 = 8198732939394532628, __1 = 1668144508664), buckets=&[leapfrog::leapmap::Bucket<u64, ying_profiler::AllocInfo>] @ 0x000000016bf31908, size_mask=4095) at leapmap.rs:1046:32
    frame #2: 0x0000000104480ca0 allocation_tests-de2cce5ea52d024e`leapfrog::leapmap::LeapMap$LT$K$C$V$C$H$C$A$GT$::insert::he4b7beadef7377c2(self=0x00000001049580e8, key=5561700864, value=(__0 = 8198732939394532628, __1 = 1668144508664)) at leapmap.rs:369:19
    frame #3: 0x000000010445c1dc allocation_tests-de2cce5ea52d024e`_$LT$ying_profiler..YingProfiler$u20$as$u20$core..alloc..global..GlobalAlloc$GT$::realloc::_$u7b$$u7b$closure$u7d$$u7d$::haa9806d619e34592(tl_state=0x000000013af127e8) at lib.rs:511:29
    frame #4: 0x0000000104469074 allocation_tests-de2cce5ea52d024e`std::thread::local::LocalKey$LT$T$GT$::try_with::h6434fb8e59e19aff(self=0x0000000104924920, f={closure_env#0} @ 0x000000016bf31dc0) at local.rs:446:16
    frame #5: 0x0000000104468a38 allocation_tests-de2cce5ea52d024e`std::thread::local::LocalKey$LT$T$GT$::with::hb9d8fb8bc986e5c2(self=0x0000000104924920, f=<unavailable>) at local.rs:422:9

I have attached a log of my lldb debug session, which shows that the probing code seems to keep going in a loop. It is as though it keeps finding null values but keeps going for some reason, I don't quite understand it.

ying-leapmap-probe-stuck.lldb-session.log

Any help would be really appreciated. The calling code to LeapMap is here, it is for insert with value type AllocStats which is basically a (u64, u64) type class/tuple. Perhaps there is something wrong with the way I'm defining a Value for it?

https://github.com/velvia/ying-profiler/blob/4e8cfbe9e547adb9de5750a8d7258b918e6d0ecd/src/lib.rs#L511

Note that I'm uniquely interested in this project because of the lock-free nature is important for this custom memory profiler I'm writing, or at least the lack of using traditional Mutex/locks, plus the freedom to allocate using alternative (in my case, the base System) allocators.

velvia avatar Nov 11 '22 17:11 velvia